Thanks for starting this discussion. In general, I agree with you that a java-based testing framework is better than the bash-based. It will help a lot for the commons and utilities.
Since I am trying to add a new bash-based Kubernetes HA test, I have some quick questions. * I am not sure where the java-based framework is. Do you mean "flink-jepsen" module or sth else? * Maybe it will be harder to run a cli command(e.g. flink run / run-application) to submit a Flink job in the java-based framework. * It will be harder to inject some operations. For example, kill the JobManager in Kubernetes. Currently, I am trying to use "kubectl exec" to do this. Best, Yang Robert Metzger <rmetz...@apache.org> 于2020年11月17日周二 下午11:36写道： > Hi all, > > Since we are currently testing the 1.12 release, and potentially adding > more automated e2e tests, I would like to bring up our end to end tests for > discussion. > > Some time ago, we introduced a Java-based testing framework, with the idea > of replacing the current bash-based end to end tests. > Since the introduction of the java-based framework, more bash tests were > actually added, making a future migration even harder. > > *For that reason, I would like to propose that we are stopping to add any > new bash end to end tests to Flink. All new end to end tests must be > written in Java and rely on the existing testing framework.* > > For the 1.13 release, I'm trying to find some time to revisit potential > improvements for the existing java e2e framework (such as using Docker > images everywhere), as well as a migration plan for the existing bash > tests. We have a large number of bash e2e tests that are just parameterized > differently. If we would start migrating them to Java, we could move a > larger proportion of tests over to the new Java framework, and tackle the > more involved bash tests later (kerberized yarn, kubernetes, ...). > > Let me know what you think! > > Best, > Robert > > > PS: If you are wondering why I'm bringing this up now: I'm spending quite a > lot of time trying to figure out really hard to debug issues with our bash > testing infra. > Also, it is very difficult to introduce something generic for all tests > (such as a test-timeout, using docker as the preferred deployment method > etc.) since the tests often don't share common tooling. > Speaking about tooling: there are a lot of utilities everywhere, sometimes > duplicated, with different features / stability etc. > I believe bash is not the right tool for a project this size (in terms of > developers and lines of code) >
