On Mon, Feb 20, 2017 at 3:10 PM Cleber Rosa <cr...@redhat.com> wrote:
> > On 02/20/2017 08:46 AM, Andrei Stepanov wrote: > > It depends. > > > > There could be a scenario where necessary to run ~ 200 tests at once. > > If first bad-by-design test forgets/failed to cleanup, then some > > sequential tests also will fail. > > It is a question about usability. > > > > Andrei, > > I've personally thought about this some time ago. Right now, Avocado > tries to "isolate" tests from each other by using separate processes. > This is a rather weak isolation, but still better than what many other > test runners offer. > > I believe that having stronger isolation at a test level would indeed be > really useful. Running each test on a separate and disposable > "execution environment" (container, vm with snapshot, physical machine > that gets reloaded) would be really powerful. > > Still, we don't have that feature at this time... > Indeed having a more isolated execution environment for tests would be powerful. The question is, what should avocado do by default, since the more isolated the environment gets, the more overhead it adds to test execution. I remember some time ago someone was complaining that avocado tests were executed on a separate subprocess, as having everything executed under the same process is faster, and indeed what unittests and pytest does by default (there is a multiprocess based pytest plugin). Which makes me think if we should have configurable levels of isolation: 1 - Fast - everything gets executed under the main avocado process 2 - Default - Tests are executed with multiprocess 3 - Tests are executed in docker containers 4 - Tests are executed in virtual machines with snapshots One of the problems to make this 'transparent' is the setup and dependencies necessary to get 3 and 4 working out of the box.