While the Cypress team suggests taking advantage of stubs where you can, especially during development, we would definitely be able to test real endpoints [1]. It can be used exactly like how Protractor is used, but with the many benefits and features it provides [2]. Cypress also offers tools for unit testing [3], which I think may be causing confusion as to what exactly the library does. Cypress' main focus is e2e tests, but because of its architecture, it can be used for all types of tests.
I agree with everything you mentioned, Mike. I think our approach now is fine, but in the future I do think it's worth considering the Cypress team's suggestions for when and when not to stub, but there are no hard and fast rules [4][5]. I currently have a branch available on my fork where I've migrated over some e2e tests from Protractor to Cypress. With the exception of a little code cleanup, these tests perform the same steps as they do with Protractor. I have yet to include instructions in the README or include an npm script, but if anyone wants to see it in action they can do the following: - download this branch: https://github.com/sardell/metron/tree/METRON-1648, - run `npm ci` from meron-alerts, - start the e2e test server, - run `./node_modules/.bin/cypress open` - start a single test by clicking on a file name in the Cypress user interface, or run them all by clicking the play button. I'll try to send some sort of benchmarks when I get a chance to show the speed difference between the two libraries. [1] https://docs.cypress.io/api/commands/request.html [2] https://www.cypress.io/features/ [3] https://docs.cypress.io/guides/guides/stubs-spies-and-clocks.html [4] https://docs.cypress.io/guides/guides/network-requests.html#Testing-Strategies . [5] https://docs.cypress.io/guides/getting-started/testing-your-app.html#Stubbing-the-Server On Thu, Sep 20, 2018 at 12:09 AM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Shane, > > Can you elaborate on the testing model you're proposing? I looked through > the overview and some of the documentation. As far as I can tell, this > would effectively be and e2e test for the UI *only*, so we would be missing > testing the actual integration points with the REST API or any other > potential endpoints. > > 1. Are you proposing we migrate all existing e2e tests, including those > that currently hit Elasticsearch? > 2. Would shifting to Cypress mean that all e2e tests would be isolated > to only what is rendered via the browser? i.e. our e2e suite is no > longer > testing integration to a backend? > > My assumption with the term e2e testing is that you are testing an entire > vertical slice with no substantive mock/stub/fake/spy/dummy [1] in the way > except for maybe some strategic cross-cutting concerns. It sounds like > Cypress does NOT mean full e2e. My initial reaction to this is that there's > a place for both forms of testing. If Cypress would help UI developers work > on incremental changes, similar to how unit tests via JUnit help Java > developers iterate on features, then I think that's great. I'm all for > that! But unit tests are only one form of testing - we also do integration > testing, which can flex multiple classes/components together, as well as > more broad stack integration/functional testing that verifies everything > works when integrated together. Generally speaking, total # of unit tests > > # of integration tests > # functional/acceptance tests. I think we should > carve out and define a testing approach for each. Can you elaborate a bit > on your vision for how to manage the test interactions, or lack thereof, > with the REST API as an integration endpoint? [2] > > At the time the write-up James shared was written, it appears that Cypress > was not yet open source. Now, it's MIT license - > https://github.com/cypress-io/cypress/blob/develop/LICENSE.md. > > Mike > > 1. > > https://martinfowler.com/articles/mocksArentStubs.html#TheDifferenceBetweenMocksAndStubs > 2. https://martinfowler.com/articles/practical-test-pyramid.html#UiTests > > > On Wed, Sep 19, 2018 at 8:47 AM James Sirota <jsir...@apache.org> wrote: > > > This article comparing the two is not favorable for Cypress. Are any of > > these concerns relevant to us? If not, then I think Cypress is fine > > > > > > > https://hackernoon.com/cypress-io-vs-protractor-e2e-testing-battle-d124ece91dc7 > > > > >
