Re: [gem5-dev] Improved regression categorisation

2014-12-23 Thread Steve Reinhardt via gem5-dev
Thanks for the clarification, Andreas. Yes, it's a good step; thanks for doing it. Steve On Tue, Dec 23, 2014 at 12:55 AM, Andreas Hansson via gem5-dev < gem5-dev@gem5.org> wrote: > > Hi Steve, > > The 00.hello tests are below 10 seconds and have too high SNR to even make > it into my report :-)

Re: [gem5-dev] Improved regression categorisation

2014-12-23 Thread Andreas Hansson via gem5-dev
Hi Steve, The 00.hello tests are below 10 seconds and have too high SNR to even make it into my report :-), so yes you are right in that they are included in the ‘short’ regressions. This is definitely an intermediate step, but in any case we benefit from having a more sensible classification. T

Re: [gem5-dev] Improved regression categorisation

2014-12-22 Thread Steve Reinhardt via gem5-dev
Sounds reasonable to me. I'm not too particular about the naming. I am surprised that even the o3 "hello world" tests wouldn't be < 180 seconds though. It would be nice to have the quick/short/zippy/whatever test category exercise o3 at least a little bit. As far as composing regression paths,

Re: [gem5-dev] Improved regression categorisation

2014-12-22 Thread Gabe Black via gem5-dev
I mean quick, medium, slow, not quick, medium, fast. On Mon, Dec 22, 2014 at 12:44 PM, Gabe Black wrote: > I complained about those names a long time ago, and I still think they > aren't very good. "quick" and "long" aren't really on the same scale, to > start with. Something can be quick (a rat

Re: [gem5-dev] Improved regression categorisation

2014-12-22 Thread Gabe Black via gem5-dev
I complained about those names a long time ago, and I still think they aren't very good. "quick" and "long" aren't really on the same scale, to start with. Something can be quick (a rate) and still take a long time. Medium is very generic and so isn't on a different axis, but since the others aren'

[gem5-dev] Improved regression categorisation

2014-12-22 Thread Andreas Hansson via gem5-dev
Hi all, At the moment we run roughly 120 regressions, and divide them into quick and long somewhat arbitrarily. Anyone doing active development and using quick as their “quick” way of checking that nothing is broken has to wait more than 10 minutes for some of these regressions to finish, which