Thanks Matthias, I have created sub-tasks for the Table SQL related bash-based e2e tests.
Best, Jark On Fri, 27 Nov 2020 at 21:25, Matthias Pohl <matth...@ververica.com> wrote: > Thanks Robert for pushing this. +1 for creating java-based e2e tests. > > In the engine team, we decided to work towards the goal of migrating the > bash-based e2e tests to Java/Docker. We plan to migrate the existing > bash-based e2e tests located in the Engine team's component space > step-by-step. I created an umbrella Jira issue [1] to collect and document > the migration efforts. Feel free to do the same by creating subtasks under > [1]. > > Best, > Matthias > > [1] https://issues.apache.org/jira/browse/FLINK-20392 > > On Thu, Nov 19, 2020 at 8:02 AM Yun Tang <myas...@live.com> wrote: > > > +1 for java-based E2E test as bash scripts lack of the power to handle > > more complicated cases. > > > > For the docker image improvement, I think we should be more cautious as > > developers at China might suffer with the network issue, or at least we > > should ensure or find some guides to speed up the image downloading. > > > > Best > > Yun Tang > > ________________________________ > > From: Xingbo Huang <hxbks...@gmail.com> > > Sent: Thursday, November 19, 2020 12:09 > > To: dev <dev@flink.apache.org> > > Subject: Re: [DISCUSS] Stop adding new bash-based e2e tests to Flink > > > > Big +1 for java-based e2e tests. Currently PyFlink related tests each > take > > ~15minutes in bash e2e tests because we are using a secured YARN cluster > > which is the only convenient way of starting a YARN cluster in the bash > e2e > > tests. I think if we migrate these tests to the java-based testing > > framework, we will start a Yarn Cluster more conveniently, which will > > greatly reduce our testing time. > > > > Best, > > Xingbo > > > > Rui Li <lirui.fu...@gmail.com> 于2020年11月19日周四 上午10:47写道： > > > > > Big +1 to java-based e2e tests. It'll be much easier to write/debug > these > > > tests. > > > > > > On Wed, Nov 18, 2020 at 9:44 PM Leonard Xu <xbjt...@gmail.com> wrote: > > > > > > > +1 to stop using bash scripts, > > > > and I also have experienced the bash scripts that is really hard to > > > > maintain and debug, thanks @Robert for the great work again. > > > > > > > > I think testcontainers is a nice candidate. > > > > > > > > Best, > > > > Leonard > > > > > > > > > 在 2020年11月18日，19:46，Aljoscha Krettek <aljos...@apache.org> 写道： > > > > > > > > > > +1 > > > > > > > > > > And I want to second Arvid's mention of testcontainers [1]. > > > > > > > > > > [1] https://www.testcontainers.org/ > > > > > > > > > > On 18.11.20 10:43, Yang Wang wrote: > > > > >> Thanks till and Jark for sharing the information. > > > > >> I am also +1 for this proposal and glad to wire the new introduced > > K8s > > > > HA > > > > >> e2e tests to java based framework. > > > > >> Best, > > > > >> Yang > > > > >> Jark Wu <imj...@gmail.com> 于2020年11月18日周三 下午5:23写道： > > > > >>> +1 to use the Java-based testing framework and +1 for using > docker > > > > images > > > > >>> in the future. > > > > >>> IIUC, the Java-based testing framework refers to the > > > > >>> `flink-end-to-end-tests-common` module. > > > > >>> The java-based framework helped us a lot when debugging the > > unstable > > > > e2e > > > > >>> tests. > > > > >>> > > > > >>> Best, > > > > >>> Jark > > > > >>> > > > > >>> On Wed, 18 Nov 2020 at 14:42, Yang Wang <danrtsey...@gmail.com> > > > wrote: > > > > >>> > > > > >>>> Thanks for starting this discussion. > > > > >>>> > > > > >>>> In general, I agree with you that a java-based testing framework > > is > > > > >>> better > > > > >>>> than the bash-based. It will > > > > >>>> help a lot for the commons and utilities. > > > > >>>> > > > > >>>> Since I am trying to add a new bash-based Kubernetes HA test, I > > have > > > > some > > > > >>>> quick questions. > > > > >>>> * I am not sure where the java-based framework is. Do you mean > > > > >>>> "flink-jepsen" module or sth else? > > > > >>>> * Maybe it will be harder to run a cli command(e.g. flink run / > > > > >>>> run-application) to submit a Flink job in the java-based > > framework. > > > > >>>> * It will be harder to inject some operations. For example, kill > > the > > > > >>>> JobManager in Kubernetes. Currently, I > > > > >>>> am trying to use "kubectl exec" to do this. > > > > >>>> > > > > >>>> > > > > >>>> Best, > > > > >>>> Yang > > > > >>>> > > > > >>>> Robert Metzger <rmetz...@apache.org> 于2020年11月17日周二 下午11:36写道： > > > > >>>> > > > > >>>>> Hi all, > > > > >>>>> > > > > >>>>> Since we are currently testing the 1.12 release, and > potentially > > > > adding > > > > >>>>> more automated e2e tests, I would like to bring up our end to > end > > > > tests > > > > >>>> for > > > > >>>>> discussion. > > > > >>>>> > > > > >>>>> Some time ago, we introduced a Java-based testing framework, > with > > > the > > > > >>>> idea > > > > >>>>> of replacing the current bash-based end to end tests. > > > > >>>>> Since the introduction of the java-based framework, more bash > > tests > > > > >>> were > > > > >>>>> actually added, making a future migration even harder. > > > > >>>>> > > > > >>>>> *For that reason, I would like to propose that we are stopping > to > > > add > > > > >>> any > > > > >>>>> new bash end to end tests to Flink. All new end to end tests > must > > > be > > > > >>>>> written in Java and rely on the existing testing framework.* > > > > >>>>> > > > > >>>>> For the 1.13 release, I'm trying to find some time to revisit > > > > potential > > > > >>>>> improvements for the existing java e2e framework (such as using > > > > Docker > > > > >>>>> images everywhere), as well as a migration plan for the > existing > > > bash > > > > >>>>> tests. We have a large number of bash e2e tests that are just > > > > >>>> parameterized > > > > >>>>> differently. If we would start migrating them to Java, we could > > > move > > > > a > > > > >>>>> larger proportion of tests over to the new Java framework, and > > > tackle > > > > >>> the > > > > >>>>> more involved bash tests later (kerberized yarn, kubernetes, > > ...). > > > > >>>>> > > > > >>>>> Let me know what you think! > > > > >>>>> > > > > >>>>> Best, > > > > >>>>> Robert > > > > >>>>> > > > > >>>>> > > > > >>>>> PS: If you are wondering why I'm bringing this up now: I'm > > spending > > > > >>>> quite a > > > > >>>>> lot of time trying to figure out really hard to debug issues > with > > > our > > > > >>>> bash > > > > >>>>> testing infra. > > > > >>>>> Also, it is very difficult to introduce something generic for > all > > > > tests > > > > >>>>> (such as a test-timeout, using docker as the preferred > deployment > > > > >>> method > > > > >>>>> etc.) since the tests often don't share common tooling. > > > > >>>>> Speaking about tooling: there are a lot of utilities > everywhere, > > > > >>>> sometimes > > > > >>>>> duplicated, with different features / stability etc. > > > > >>>>> I believe bash is not the right tool for a project this size > (in > > > > terms > > > > >>> of > > > > >>>>> developers and lines of code) > > > > >>>>> > > > > >>>> > > > > >>> > > > > > > > > > > > > > > > > > > > -- > > > Best regards! > > > Rui Li > > > > > > > > -- > > Matthias Pohl | Engineer > > Follow us @VervericaData Ververica <https://www.ververica.com/> > > -- > > Join Flink Forward <https://flink-forward.org/> - The Apache Flink > Conference > > Stream Processing | Event Driven | Real Time > > -- > > Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany > > -- > Ververica GmbH > Registered at Amtsgericht Charlottenburg: HRB 158244 B > Managing Directors: Yip Park Tung Jason, Jinwei (Kevin) Zhang, Karl Anton > Wehner >
