On Thu, Apr 06, 2017 at 12:38:31PM -0400, Cleber Rosa wrote: [...] > >>> Do you have any suggestions for making the test result output > >>> from those scripts easily consumable by the Avocado test runner? > >>> > >> > >> If those tests are treated as SIMPLE tests, then all of their output > >> will automatically be added to the Avocado logs. > >> > >> Now, I do understand what your hopes are (or were after my possibly > >> disappointing recommendations :). Ideally, Avocado would be able to > >> find the tests available on your standalone scripts[6], would know how > >> to run the individual tests[7]. Right now, this requires writing a > >> plugin. I can see some ideas for a "Python universal" plugin here, that > >> would find other hints of tests in plain Python source code. If you > >> think that makes sense, let's discuss it further. > >> > > Actually we already have `robot` plugin which is able to discover > > `robot`-like tests. How about creating a `unittest` loader, which would > > do the `avocado-find-unittest` and discover them individually? By > > default nothing would change as the `.py` files would be recognized as > > instrumented/simple tests, but when you change the order of loaders it'd > > detect them as unittests: > > > > Eduardo mentioned that he'd eventually get rid of the unittest > dependency. That's why I was suggesting/brainstorming about an even > more generic way of providing hints that there are tests on a Python > source file.
I planned to get rid of the unittest dependency because it did not seem useful to me. But if it provides Avocado test discovery ability for free, I would probably keeping it. Now, another problem is that I wish a test granuality that requires running QEMU at least once during the test discovery phase (e.g. one test case for each machine-type/device/CPU). Maybe this will break some assumptions in Avocado? Are tests able to get a few input parameters during thest discovery? e.g. I want to tell the test script/module the list of QEMU binaries I want to test. Today I use an environment variable or command-line parameter for that. -- Eduardo