Hi Dominik, yes this is a good point. There is already a wiki page to collect descriptions for tests [1]. This can be used to collect features that should be tested. I will also extend a section how to use the test framework.
To write the tests it is super helpful when there is already a script with the workflow that defines what should be tested, as well as the values for the input parameters. We can also collect those in the wiki, as I did it for the third-party tests (e.g. Kafka, MQTT) [2]. With such information it is much easier to write the test workflow in typescript. So, if anyone has suggestions what should be tested, feel free to add a description in the wiki. I am also in favor of using the upcoming release to write some tests to increase the test coverage. Philipp [1] https://cwiki.apache.org/confluence/display/STREAMPIPES/Description+for+E2E+Tests [2] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=184616129 Am 30.11.21, 17:16 schrieb "Dominik Riemer" <[email protected]>: Hi Philipp, very cool! Running the workflows once daily sounds good to me. What about having a short documentation in the developer wiki so that everyone knows how to write tests? It would be awesome to have more test coverage for the whole system and I think writing a few new tests is also a good start for new contributors. Dominik On 2021/11/30 14:35:36 Philipp Zehnder wrote: > Hi all, > > > > we were working on automated e2e tests, especially to ease the manual testing effort of all the different components and algorithms within StreamPipes. > > Since we now have a single repository, it is also possible to run all the tests directly in GitHub actions. > > > > I prepared a workflow in [1] that builds StreamPipes and runs the tests. > > The tests are defined in [2]. For the naming convention I decided to use the suffix ‘.spec.ts’ for tests and ‘smoke.spec.ts’ for smoke tests. > > The main reason for this distinction is the time the tests require to execute, especially testing all the processors takes quite some time. > > Some tests only have a .ts suffix. They are not included into one of the test suites. The reason is that they require some third-party dependencies (e.g. mysql). > > Those tests can be triggered manually, but it must be ensured that the dependencies are running. > > > Smoke tests (*.smoke.spec.ts) to early find bugs (e.g. on commit / on PR) > All tests (*.spec.ts) running all e2e tests > Manual tests (*.ts) must be triggered manually when running cypress locally > > > My suggestion would be to run the smoke tests on each PR and have a workflow that runs all the tests once a day. > > What is your opinion on that? > > We can also discuss about the naming and which tests should be included in which test suite. > > > > You can run the test via npm locally [3] or use the provided workflow for GitHub actions > > > > Philipp > > > > > > [1] https://github.com/apache/incubator-streampipes/blob/dev/.github/workflows/cypress-test.yml > > [2] https://github.com/apache/incubator-streampipes/tree/dev/ui/cypress/tests > > [3] https://github.com/apache/incubator-streampipes/blob/dev/ui/package.json > > > > > >
