I am all-in for markers. I think we should start with small set of useful markers, which should have a useful purpose from the beginning and implement them first - to learn how useful they are (before we decide on full set of markers). Otherwise maintaining those markers will become a fruitless "chore" and it might be abandoned.
So my proposal is to agree the first top cases we want to handle with markers and then define/apply the markers accordingly: Those are my three top priorities (from most important to least): - Splitting out the Integration tests (and updating Breeze) so that you choose which integration you start when you start Breeze rather than start them all. - DB separation so that we do not repeat non-DB tests on all Databases. - Proper separation of Kubernetes tests (They are now filtered out based on skipif/env variables. J. On Tue, Dec 10, 2019 at 1:32 PM Tomasz Urbaszek <tomasz.urbas...@polidea.com> wrote: > Hi everyone, > > Since we run our tests using pytest we are able to use test markers [1]. > Using them will give > use some useful things: > - additional information of test type (ex. when used for system test) > - easy way to select test by types (ex. pytest -v -m "not system") > - way to split our test suite in more effective way (no need to run all > tests on 3 backends) > > I would like to discuss what "official" marks would we like to use. As a > base I would suggests > to mark tests as: > - system - tests that need the outside world to be successful (ex. GCP > system tests) > - db[postgres, sqlite, mysql] - tests that require database to be > successful, in other words, > tests that create some db side effects > - integration - tests that requires some additional resources like > Cassandra or Kubernetes > > All other, unmarked tests would be treated as "pure" meaning that they have > no side effects > (at least on database level). > > What do you think about this? Does anyone have some experience with using > markers in > such a big project? > > [1] http://doc.pytest.org/en/latest/example/markers.html > > > Bests, > Tomek Urbaszek > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
