Ducktests were merged, thanks to everyone involved! Co-authored-by: Nikolay Izhikov Co-authored-by: Ivan Daschinskiy Co-authored-by: Maksim Timonin Co-authored-by: Vladsz83 Co-authored-by: oleg-ostanin Co-authored-by: SwirMix Co-authored-by: Sergei Ryzhov Co-authored-by: Mikhail Filatov Co-authored-by: Dmitriy Sorokin Co-authored-by: emvdovets
On Mon, Jun 28, 2021 at 6:01 PM Anton Vinogradov <[email protected]> wrote: > Seems, we have a silent agreement here :) > We also discussed the proposal with the Ignite QA Meetup participants and > found this contribution useful. > > Going to merge tomorrow, if there are no objections. > > On Mon, Jun 28, 2021 at 3:21 PM Anton Vinogradov <[email protected]> wrote: > >> With regard to the request of Ivan Daschinskiy checked that we are able >> to release code from the PR. >> Looks like everything [1] is fine, ignite-ducketests are not a part of >> the zip file. >> >> [1] >> https://ci.ignite.apache.org/buildConfiguration/Releases_ApacheIgniteMain_ReleaseBuild/6064064?buildTab=artifacts#%2Frelease-2.11.0-rc0.zip!%2Fmaven;%2Frelease-2.11.0-rc0.zip!%2Fpackages >> (you may need special permission to open this page) >> >> On Mon, Jun 28, 2021 at 11:01 AM Nikolay Izhikov <[email protected]> >> wrote: >> >>> Hello. >>> >>> As one of the authors of PR fully support merge of this module. >>> Have plans to use it during development of other features. >>> Guys, please, share your feedback. >>> >>> Do we need to improve something prior merge? >>> >>> >>> > 24 июня 2021 г., в 14:41, Denis Magda <[email protected]> написал(а): >>> > >>> > Anyway, it's much more community-friendly. Thanks, Anton! >>> > >>> > -- >>> > Denis >>> > >>> > - >>> > Denis >>> > >>> > On Thu, Jun 24, 2021 at 3:43 AM Anton Vinogradov <[email protected]> >>> wrote: >>> > >>> >> Denis, >>> >> >>> >> Unfortunately, we had some issues with this recording (low slides >>> recording >>> >> resolution, missed slides and random slides switch delay), which >>> makes it >>> >> slightly unwatchable :( >>> >> Anyway, if you brave enough, you may open the slides [1] and the >>> recording >>> >> [2] simultaneously to solve this :) >>> >> >>> >> [1] >>> >> >>> >> >>> https://go.gridgain.com/rs/491-TWR-806/images/11_IgniteSummit2021-Deep-dive-into-testing.pdf >>> >> [2] https://www.youtube.com/watch?v=uRRlGrSA3NY >>> >> >>> >> On Wed, Jun 23, 2021 at 5:40 PM Denis Magda <[email protected]> >>> wrote: >>> >> >>> >>> Congrats, Anton, that's a valuable contribution! I attended your >>> session >>> >> at >>> >>> the Ignite Summit and wonder if you should share that recording with >>> an >>> >>> English-speaking part of the community? >>> >>> >>> >>> - >>> >>> Denis >>> >>> >>> >>> On Wed, Jun 23, 2021 at 7:37 AM Anton Vinogradov <[email protected]> >>> wrote: >>> >>> >>> >>>> Folks, >>> >>>> >>> >>>> Here's the video [1] that explains the proposal in detail. >>> >>>> Feel free to ask questions here. >>> >>>> >>> >>>> [1] https://www.youtube.com/watch?v=f-i9COU5uAQ (in Russian) >>> >>>> >>> >>>> On Tue, Jun 8, 2021 at 2:51 PM Anton Vinogradov <[email protected]> >>> wrote: >>> >>>> >>> >>>>> Igniters, >>> >>>>> Let me present a framework, we developed, that allows automating >>> >> Apache >>> >>>>> Ignite testing on a real cluster. >>> >>>>> The framework was initially presented at Ignite Summit. >>> >>>>> >>> >>>>> In brief, >>> >>>>> The framework allows automating operations with any applications >>> on a >>> >>>> real >>> >>>>> cluster using ssh in a form of a python test. >>> >>>>> >>> >>>>> Features: >>> >>>>> - Ignite nodes can be started/stopped on a Docker or a real cluster >>> >>> with >>> >>>>> any custom configuration >>> >>>>> - Any AI version is supported (released or compiled from sources) >>> >>>>> - Ignite forks are also supported «out of the box» >>> >>>>> - Any other application execution is also possible, eg. we >>> >> implemented >>> >>>>> starters for Spark and Zookeeper >>> >>>>> - Cluster can be managed using the control.sh, we made this a part >>> of >>> >>> the >>> >>>>> test API >>> >>>>> - Custom Java applications can be executed remotely with/without a >>> >>>>> built-in Ignite node or a Thin client >>> >>>>> - Any ssh command can be executed remotely, and the result will be >>> >>>>> available locally (at the python test) >>> >>>>> - Network can be broken by iptables change to check communication >>> >>> issues >>> >>>>> - Tests can be executed in parallel when cluster size bigger than >>> >> tests >>> >>>>> requirements >>> >>>>> >>> >>>>> As a result, seems, we may automate any testing scenario we can >>> even >>> >>>>> imagine. >>> >>>>> >>> >>>>> Framework based on Ducktape [1] library from Kafka team, that's why >>> >> we >>> >>>>> called it Ducktests. >>> >>>>> >>> >>>>> The Ducktests were developed during work on IEP-56 [2] and already >>> >> were >>> >>>>> used during the work on IEP-45 [3]. >>> >>>>> IEP-45 measurement results examples [4.1] [4.2] were demonstrated >>> at >>> >>> the >>> >>>>> HighLoad conference last month. >>> >>>>> >>> >>>>> Code available at PR-9117 [5] and ready to be reviewed/merged. >>> >>>>> Feel free to ask questions or make proposals. >>> >>>>> >>> >>>>> [1] https://ducktape-docs.readthedocs.io/en/latest/index.html >>> >>>>> [2] >>> >>>>> >>> >>>> >>> >>> >>> >> >>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-56%3A+Distributed+environment+tests >>> >>>>> [3] >>> >>>>> >>> >>>> >>> >>> >>> >> >>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-45%3A+Crash+Recovery+Speed-Up >>> >>>>> [4.1] https://youtu.be/UZsvCNjbkww?t=1065 (in Russian) >>> >>>>> [4.2] https://youtu.be/UZsvCNjbkww?t=1609 (in Russian) >>> >>>>> [5] https://github.com/apache/ignite/pull/9117 >>> >>>>> >>> >>>> >>> >>> >>> >> >>> >>>
