On Thu, 01 May 2014 14:40:41 -0700 Andrei Alexandrescu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 5/1/14, 2:28 PM, Jason Spencer wrote: > > But it seems the key question is whether order can EVER be > > important for any reason. I for one would be willing to give up > > parallelization to get levelized tests. What are you seeing on > > your project? How do you allow tests to have dependencies and > > avoid order issues? Why is parallelization more important than > > that? > > I'll be blunt. What you say is technically sound (which is probably > why you believe it is notable) but seems to me an unnecessarily > complex engineering contraption that in all likelihood has more > misuses than good uses. I fully understand you may think I'm a > complete chowderhead for saying this; in the past I've been in your > place and others have been in mine, and it took me years to > appreciate both positions. -- Andrei It's my understanding that given how druntime is put together, it should be possible to override some of its behaviors such that you could control the order in which tests were run (the main thing lacking at this point is the fact that you can currently only control it at module-level granularity) and that that's what existing third party unit test frameworks for D do. So, I would think that we could make it so that the default test runner does things the sensible way that works for most everyone, and then anyone who really wants more control can choose to override the normal test runner to do run the tests the way that they want to. That should be essentially the way that it is now. The main question then is which features we think are sensible for everyone, and I think that based on this discussion, at this point, it's primarily 1. Make it possible for druntime to access unit test functions individually. 2. Make it so that druntime runs unit test functions in parallel unless they're marked as needing to be run in serial (probably with a UDA for that purpose). 3. Make it so that we can name unittest blocks so that stack traces have better function names in them. With those sorted out, we can look at further features like whether we want to be able to run unit tests by name (or whatever other nice features we can come up with), but we might as well start there rather than trying to come up with a comprehensive list of the features that D's unit testing facilities should have (especially since we really should be erring on the side of simple). - Jonathan M Davis