Hi, all I want to use this thread to discuss the integration tests for adapters and sinks.
Currently, there seems to be very little testing for adapters and sinks. It’s not convenient to develop and fix them. We already have the e2e tests for these third-party components[0]. But this requires starting all third-party docker containers before running e2e tests. If there are many adapters and sinks components(actually we already have), it will lead to inconvenient testing. It also seems inconvenient to develop and debug them. I wonder if there is a better way to optimize this testing approach for third-party components. I would like to propose adding the integration test for adapters and sinks. And we could still use the e2e tests to do some small smoke tests. We could add a new module called streampipes-integration-tests. In this module, we do all integration tests for all adapters and sinks and perhaps some other components. For each test, we could start the docker container of the third-party component using Testcontainers[1]. And we use the streampipes-backend to create, interact, and delete the component through the Rest protocol. We could abstract some common tests and utilities for all adapters and sinks. Using the streampipes-integration-tests, we can test a large number of adapters and sinks and be able to clean up these docker container resources in a timely manner. It also facilitates the development and debugging. It is still possible to test these components at a higher level(at the backend level). In addition, regarding the unit test, we could also add some unit tests for each adapter and sink at their corresponding modules if necessary. Please feel free to share your thoughts. And I'm interested in making it happen. [0] https://github.com/apache/incubator-streampipes/tree/dev/ui/cypress/tests/thirdparty [1] https://www.testcontainers.org/ Thanks, Zike Yang
