Hi,
Thank you for initiating this discussion and already providing a solid
foundation for integration testing for StreamPipes.
I completely agree with you and Philipp that testing (both in terms of
unit tests and integration tests) is a weak point in our codebase and
has been treated stepmotherly so far.
It would be great to have integration tests for all our adapters and
sinks.
Maybe we can start with a rather simple example and based on that
together find a way to implement integration testing for StreamPipes?
I would be happy to contribute to this effort to some extent.
Best
Tim
Am 11.10.2022 19:15 schrieb Philipp Zehnder:
Hi Zike Yang,
thanks a lot for opening the discussion. I really like this idea!
We started with the e2e tests to have some basic tests, but we
definitely need way more test coverage and any help is highly
appreciated.
To have some kind of integration or unit tests for adapters and
processors would be awesome, especially because users can provide a
wide range of input configurations.
For the e2e tests we tried to provide as much as possible via
configuration files see [1]. They describe the configuration for the
processors, the input data and the expected output data. This is
required to also test more complex patterns in the event stream. Do
you have an idea how we can realize all the user input in the
integration tests?
Maybe it is even possible to re-use those or similar configuration
files for both the integration and the e2e tests of processors. I
expect the integration tests to be much faster then the e2e tests, but
for the e2e tests we can ensure that the user can use the GUI for all
the input.
How can we deal with third party dependencies? Especially for adapters
and data sinks we require other services that must be configured
accordingly. Do you have any experience with that how to do it best?
I am really looking forward to discuss this further with you.
Cheers,
Philipp
[1]
https://github.com/apache/incubator-streampipes/tree/dev/ui/cypress/fixtures/pipelineElement
Von: Zike Yang <[email protected]>
Datum: Dienstag, 11. Oktober 2022 um 17:31
An: [email protected] <[email protected]>
Betreff: [DISCUSS] Integration test for adapters and sinks
Hi, all
I want to use this thread to discuss the integration tests for
adapters and sinks.
Currently, there seems to be very little testing for adapters and
sinks. It’s not convenient to develop and fix them. We already have
the e2e tests for these third-party components[0]. But this requires
starting all third-party docker containers before running e2e tests.
If there are many adapters and sinks components(actually we already
have), it will lead to inconvenient testing. It also seems
inconvenient to develop and debug them. I wonder if there is a better
way to optimize this testing approach for third-party components.
I would like to propose adding the integration test for adapters and
sinks. And we could still use the e2e tests to do some small smoke
tests.
We could add a new module called streampipes-integration-tests. In
this module, we do all integration tests for all adapters and sinks
and perhaps some other components. For each test, we could start the
docker container of the third-party component using Testcontainers[1].
And we use the streampipes-backend to create, interact, and delete the
component through the Rest protocol. We could abstract some common
tests and utilities for all adapters and sinks.
Using the streampipes-integration-tests, we can test a large number of
adapters and sinks and be able to clean up these docker container
resources in a timely manner. It also facilitates the development and
debugging. It is still possible to test these components at a higher
level(at the backend level).
In addition, regarding the unit test, we could also add some unit
tests for each adapter and sink at their corresponding modules if
necessary.
Please feel free to share your thoughts. And I'm interested in making
it happen.
[0]
https://github.com/apache/incubator-streampipes/tree/dev/ui/cypress/tests/thirdparty
[1] https://www.testcontainers.org/
Thanks,
Zike Yang
