I also would like to eventually have these tests automated. There are a couple hurdles to setting up our e2e tests to run with our build. I think the biggest hurdle is setting up a dedicated server with data for the e2e tests to use. I would assume this requires funding, engineering support, obfuscated data, etc. I also think we should migrate our e2e tests to Cypress first because Protractor lacks debugging tools that would make our life much easier if, for example, we had a failure in our CI build but could not reproduce locally. In addition, our current Protractor tests are brittle and extremely slow.
All that said, it seems we agree that we could add another PR checklist item in the meantime. Clarifying those e2e test instructions should be part of that task. On Mon, Oct 1, 2018 at 2:36 PM Casey Stella <ceste...@gmail.com> wrote: > I'd also like to make sure that clear instructions are provided (or linked > to) about how to run them. Also, we need to make sure the instructions are > rock-solid for running them. > Looking at > > https://github.com/apache/metron/tree/master/metron-interface/metron-alerts#e2e-tests > , > would someone who doesn't have much or any knowledge of the UI be able to > run that without assistance? > > For instance, we use full-dev, do we need to stop data from being played > into full-dev for the tests to work? > > Casey > > On Mon, Oct 1, 2018 at 8:29 AM Casey Stella <ceste...@gmail.com> wrote: > > > I'm not super keen on expanding the steps to contribute, especially in an > > avenue that should be automated. > > That being said, I think that until we get to the point of automating the > > e2e tests, it's sensible to add them to the checklist. > > So, I would support it, but I would also urge us to move forward the > > efforts of running these tests as part of the CI build. > > > > What is the current gap there? > > > > Casey > > > > On Mon, Oct 1, 2018 at 7:41 AM Shane Ardell <shane.m.ard...@gmail.com> > > wrote: > > > >> Hello everyone, > >> > >> In another discussion thread from July, I briefly mentioned the idea of > >> adding a step to the pull request checklist asking contributors to run > the > >> UI end-to-end tests. Since we aren't running e2e tests as part of the CI > >> build, it's easy for contributors to unintentionally break these tests. > >> Reminding contributors to run these tests will hopefully help catch > >> situations like this before opening a pull request. > >> > >> Does this make sense to everyone? > >> > >> Regards, > >> Shane > >> > > >
