+1 to stop using bash scripts, and I also have experienced the bash scripts that is really hard to maintain and debug, thanks @Robert for the great work again.
I think testcontainers is a nice candidate. Best, Leonard > 在 2020年11月18日，19:46，Aljoscha Krettek <aljos...@apache.org> 写道： > > +1 > > And I want to second Arvid's mention of testcontainers [1]. > > [1] https://www.testcontainers.org/ > > On 18.11.20 10:43, Yang Wang wrote: >> Thanks till and Jark for sharing the information. >> I am also +1 for this proposal and glad to wire the new introduced K8s HA >> e2e tests to java based framework. >> Best, >> Yang >> Jark Wu <imj...@gmail.com> 于2020年11月18日周三 下午5:23写道： >>> +1 to use the Java-based testing framework and +1 for using docker images >>> in the future. >>> IIUC, the Java-based testing framework refers to the >>> `flink-end-to-end-tests-common` module. >>> The java-based framework helped us a lot when debugging the unstable e2e >>> tests. >>> >>> Best, >>> Jark >>> >>> On Wed, 18 Nov 2020 at 14:42, Yang Wang <danrtsey...@gmail.com> wrote: >>> >>>> Thanks for starting this discussion. >>>> >>>> In general, I agree with you that a java-based testing framework is >>> better >>>> than the bash-based. It will >>>> help a lot for the commons and utilities. >>>> >>>> Since I am trying to add a new bash-based Kubernetes HA test, I have some >>>> quick questions. >>>> * I am not sure where the java-based framework is. Do you mean >>>> "flink-jepsen" module or sth else? >>>> * Maybe it will be harder to run a cli command(e.g. flink run / >>>> run-application) to submit a Flink job in the java-based framework. >>>> * It will be harder to inject some operations. For example, kill the >>>> JobManager in Kubernetes. Currently, I >>>> am trying to use "kubectl exec" to do this. >>>> >>>> >>>> Best, >>>> Yang >>>> >>>> Robert Metzger <rmetz...@apache.org> 于2020年11月17日周二 下午11:36写道： >>>> >>>>> Hi all, >>>>> >>>>> Since we are currently testing the 1.12 release, and potentially adding >>>>> more automated e2e tests, I would like to bring up our end to end tests >>>> for >>>>> discussion. >>>>> >>>>> Some time ago, we introduced a Java-based testing framework, with the >>>> idea >>>>> of replacing the current bash-based end to end tests. >>>>> Since the introduction of the java-based framework, more bash tests >>> were >>>>> actually added, making a future migration even harder. >>>>> >>>>> *For that reason, I would like to propose that we are stopping to add >>> any >>>>> new bash end to end tests to Flink. All new end to end tests must be >>>>> written in Java and rely on the existing testing framework.* >>>>> >>>>> For the 1.13 release, I'm trying to find some time to revisit potential >>>>> improvements for the existing java e2e framework (such as using Docker >>>>> images everywhere), as well as a migration plan for the existing bash >>>>> tests. We have a large number of bash e2e tests that are just >>>> parameterized >>>>> differently. If we would start migrating them to Java, we could move a >>>>> larger proportion of tests over to the new Java framework, and tackle >>> the >>>>> more involved bash tests later (kerberized yarn, kubernetes, ...). >>>>> >>>>> Let me know what you think! >>>>> >>>>> Best, >>>>> Robert >>>>> >>>>> >>>>> PS: If you are wondering why I'm bringing this up now: I'm spending >>>> quite a >>>>> lot of time trying to figure out really hard to debug issues with our >>>> bash >>>>> testing infra. >>>>> Also, it is very difficult to introduce something generic for all tests >>>>> (such as a test-timeout, using docker as the preferred deployment >>> method >>>>> etc.) since the tests often don't share common tooling. >>>>> Speaking about tooling: there are a lot of utilities everywhere, >>>> sometimes >>>>> duplicated, with different features / stability etc. >>>>> I believe bash is not the right tool for a project this size (in terms >>> of >>>>> developers and lines of code) >>>>> >>>> >>> >
