On Wed, 30 Apr 2014 22:32:33 -0700 Andrei Alexandrescu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 4/30/14, 10:01 PM, Jonathan M Davis via Digitalmars-d wrote: > > I'm all for parallelizing all unittest blocks that are pure, as > > doing so would be safe, but I think that we're making a big mistake > > if we try and insist that all unittest blocks be able to be run in > > parallel. Any that aren't pure are not guaranteed to be > > parallelizable, and any which access system resources or other > > global, mutable state stand a good chance of breaking. > > There are a number of assumptions here: (a) most unittests that can > be effectively parallelized can be actually inferred (or declared) as > pure; (b) most unittests that cannot be inferred as pure are likely > to break; (c) it's a big deal if unittests break. I question all of > these assumptions. In particular I consider unittests that depend on > one another an effective antipattern that needs to be eradicated. Even if they don't depend on each other, they can depend on the system. std.file's unit tests will break if we parallelize them, because it operates on files and directories, and many of those tests operate on the same temp directories. That can be fixed by changing the tests, but it will break the tests. Other tests _can't_ be fixed if we force them to run in parallel. For instance, some of std.datetime's unit tests set the local time zone of the system in order to test that LocalTime works correctly. That sets it for the whole program, so all threads will be affected even if they're running other tests. Right now, this isn't a problem, because those tests set the timezone at their start and reset it at their end. But if they were made to run in parallel with any other tests involving LocalTime, there's a good chance that those tests would have random test failures. They simply can't be run in parallel due to a system resource that we can't make thread-local. So, regardless of how we want to mark up unittest blocks as parallelizable or not parallelizable (be it explicit, implict, using pure, or using something else), we do need a way to make it so that a unittest block is not run in parallel with any other unittest block. We can guarantee that pure functions can safely be run in parallel. We _cannot_ guarantee that impure functions can safely be run in parallel. I'm sure that many impure unittest functions could be safely run in parallel, but it would require that the programmer verify that if we don't want undefined behavior - just like programmers have to verify that @system code is actually @safe. Simply running all unittest blocks in parallel is akin to considering @system code @safe in a particular piece of code simply because by convention that code should be @safe. pure allows us to detect guaranteed, safe parallelizability. If we want to define some other way to make it so a unittest block can be marked as parallelizable regardless of purity, then fine. But automatically parallelizing impure functions means that we're going to have undefined behavior for those unittest functions, and I really think that that is a bad idea - in addition to the fact that some unittest blocks legitimately cannot be run in parallel due to the use of system resources, so parallelizing them _will_ not only break them but make them impossible to write in a way that's not broken without adding mutexes to the unittest blocks to stop the test runner from running them in parallel. And IMHO, if we end up having to do that anywhere, we've done something very wrong with how unit tests work. - Jonathan M Davis