On 5/23/19 7:11 PM, Aleksandar Markovic wrote: > On May 23, 2019 3:45 PM, "Cleber Rosa" <cr...@redhat.com> wrote: >>> From: "Aleksandar Markovic" <aleksandar.m.m...@gmail.com> >>> On May 22, 2019 11:46 PM, "Cleber Rosa" <cr...@redhat.com> wrote: >>>>> From: "Eduardo Habkost" <ehabk...@redhat.com> >>>>> >>>>> Until we actually have a mechanism to exclude the test case on >>>>> travis-ci, I will remove patch 4/4 from the queue. Aleksandar, >>>>> please don't merge patch 4/4 yet or it will break travis-ci. >>>>> >>>>> Cleber, Wainer, is it already possible to make "avocado run" skip >>>>> tests tagged with "slow"? >>>>> >>>> >>>> The mechanism exists, but we haven't tagged any test so far as slow. >>>> >>> >>> Cleber, >>> >>> For the test from patch 4/4, there is no dilemma - it should be in the >>> “slow” group, as Philippe envisioned and said, so that it is not > humpered >>> with stricter requirements for “fast” (default) group. Could you > explain us >>> how to do it, so that we can hopefully finally proceed? >>> >> >> Hi Aleksandar, >> >> The point is that there's no "group" definition at this point. This is > the >> core of the discussion. >> >> I think we're close to converging to something simple and effective. > Please >> let us know what you think of the proposals given. >> >> Thanks! >> - Cleber. >> > > Cleber, hi. > > Thanks for responding. > > My views are very similar to Philippe's, but I will provide you with more > details of our (mips) perspective. > > As far as black/whitelist issues that is a moot point for us - we only want > to be able to have a way to tag a test within the test itself (so, without > updating some common files, external lists,etc.) > > In general, we would like to have a test environment where we would be able > to test what WE deem suitable for us, without feeling that we bother you or > anybody else, or that we are bothered by others. > > Let me give you a little extreme example: Let's say we want a complex test > that downloads components from let's say fifty internet location, executes > zillion test cases, and last two days. I wouldn't like anybody to ask me > “Why would you that?” or tell me “You can't do this.” or say “No, we did > not anticipate such tests, patch rejected.” I (we, people from mips) should > be able to define what I (we) need.
Maybe we can use subdirectory like we do for the TCG tests (Aleksandar maintains tests/tcg/mips/). We should try to keep contribution upstream, so good idea/pattern can be reused by others. What I'd like to have with those tests is, at least: 1/ we don't need to run all the tests (but there is a set of 'quick' tests we can run on daily basis) 2/ maintainers can run their default tests easily (using a combination of Avocado tags) 3/ if a developer working on the PCI subsystem has to modify the MIPS subsystem (for example), he should be able to run the MIPS tests before sending his series. > Having such test would be a big deal for me, not only that I could run it > manually or automatically every weekend, but I could ask submitters of > critical changes: “Did you run this test that we have in Avocado dir?”, > without specifying test details, procedures, etc. All this is a BIG deal > for me. > > On the other hand, I agree that certain group of tests (envisioned for > daily or so Travis CI) should have some stricter limitations and structure. > But right now I feel humpered by it, and this is counterproductive. > > So, we want freedom, responsibility and ownersheep of our tests. Please > give us the opportunity to get down on business and start writing tests and > start testing. > > Yours, > Aleksandar