Hello, Branch with duck tape created - https://github.com/apache/ignite/tree/ignite-ducktape
Any who are willing to contribute to PoC are welcome. > 21 мая 2020 г., в 22:33, Nikolay Izhikov <nizhikov....@gmail.com> написал(а): > > Hello, Denis. > > There is no rush with these improvements. > We can wait for Maxim proposal and compare two solutions :) > >> 21 мая 2020 г., в 22:24, Denis Magda <dma...@apache.org> написал(а): >> >> Hi Nikolay, >> >> Thanks for kicking off this conversation and sharing your findings with the >> results. That's the right initiative. I do agree that Ignite needs to have >> an integration testing framework with capabilities listed by you. >> >> As we discussed privately, I would only check if instead of >> Confluent's Ducktape library, we can use an integration testing framework >> developed by GridGain for testing of Ignite/GridGain clusters. That >> framework has been battle-tested and might be more convenient for >> Ignite-specific workloads. Let's wait for @Maksim Shonichev >> <mshonic...@gridgain.com> who promised to join this thread once he finishes >> preparing the usage examples of the framework. To my knowledge, Max has >> already been working on that for several days. >> >> - >> Denis >> >> >> On Thu, May 21, 2020 at 12:27 AM Nikolay Izhikov <nizhi...@apache.org> >> wrote: >> >>> Hello, Igniters. >>> >>> I created a PoC [1] for the integration tests of Ignite. >>> >>> Let me briefly explain the gap I want to cover: >>> >>> 1. For now, we don’t have a solution for automated testing of Ignite on >>> «real cluster». >>> By «real cluster» I mean cluster «like a production»: >>> * client and server nodes deployed on different hosts. >>> * thin clients perform queries from some other hosts >>> * etc. >>> >>> 2. We don’t have a solution for automated benchmarks of some internal >>> Ignite process >>> * PME >>> * rebalance. >>> This means we don’t know - Do we perform rebalance(or PME) in 2.7.0 faster >>> or slower than in 2.8.0 for the same cluster? >>> >>> 3. We don’t have a solution for automated testing of Ignite integration in >>> a real-world environment: >>> Ignite-Spark integration can be taken as an example. >>> I think some ML solutions also should be tested in real-world deployments. >>> >>> Solution: >>> >>> I propose to use duck tape library from confluent (apache 2.0 license) >>> I tested it both on the real cluster(Yandex Cloud) and on the local >>> environment(docker) and it works just fine. >>> >>> PoC contains following services: >>> >>> * Simple rebalance test: >>> Start 2 server nodes, >>> Create some data with Ignite client, >>> Start one more server node, >>> Wait for rebalance finish >>> * Simple Ignite-Spark integration test: >>> Start 1 Spark master, start 1 Spark worker, >>> Start 1 Ignite server node >>> Create some data with Ignite client, >>> Check data in application that queries it from Spark. >>> >>> All tests are fully automated. >>> Logs collection works just fine. >>> You can see an example of the tests report - [4]. >>> >>> Pros: >>> >>> * Ability to test local changes(no need to public changes to some remote >>> repository or similar). >>> * Ability to parametrize test environment(run the same tests on different >>> JDK, JVM params, config, etc.) >>> * Isolation by default so system tests are as reliable as possible. >>> * Utilities for pulling up and tearing down services easily in clusters in >>> different environments (e.g. local, custom cluster, Vagrant, K8s, Mesos, >>> Docker, cloud providers, etc.) >>> * Easy to write unit tests for distributed systems >>> * Adopted and successfully used by other distributed open source project - >>> Apache Kafka. >>> * Collect results (e.g. logs, console output) >>> * Report results (e.g. expected conditions met, performance results, etc.) >>> >>> WDYT? >>> >>> [1] https://github.com/nizhikov/ignite/pull/15 >>> [2] https://github.com/confluentinc/ducktape >>> [3] https://ducktape-docs.readthedocs.io/en/latest/run_tests.html >>> [4] https://yadi.sk/d/JC8ciJZjrkdndg >
