I complained about those names a long time ago, and I still think they aren't very good. "quick" and "long" aren't really on the same scale, to start with. Something can be quick (a rate) and still take a long time. Medium is very generic and so isn't on a different axis, but since the others aren't lined up it's not as clear as it could be. I would suggest either:
short, medium, long or quick, medium, fast Preferably the first. We have another collection of options the second would collide with, namely fast, opt, debug, etc. If somebody new came along and saw there were fast/quick and opt/long regressions, it wouldn't be obvious what that meant. I also think it's not easy to compose one of those regression paths since I can never remember what all the parts are or what order they go in and it's not documented anywhere obvious. That's a separate problem though. Gabe On Mon, Dec 22, 2014 at 2:39 AM, Andreas Hansson via gem5-dev < gem5-dev@gem5.org> wrote: > Hi all, > > At the moment we run roughly 120 regressions, and divide them into quick > and long somewhat arbitrarily. Anyone doing active development and using > quick as their “quick” way of checking that nothing is broken has to wait > more than 10 minutes for some of these regressions to finish, which seems a > bit of a stretch. It turns out the actual regression run times follow an > exponential distribution, ranging from a few seconds up to >10k seconds > (almost 3 hours). I propose we also start using medium (mentioned in a few > places), and use a slightly more structured approach in dividing them up > into quick, medium and long. > > Here is what I propose: > > Quick – anything below 180 seconds, resulting in roughly 40 regressions > across all ISAs. The turn around for a quick regression run for NULL, > ALPHA, ARM and X86 (what I would deem the minimum to run) should thus be > below 5 minutes of wall-clock time. Note that there are plenty > configurations not covered by this (o3, realview64 etc). > > Medium – anything above 180 seconds, but below 1800 seconds, also > resulting in roughly 40 regressions. > > Long – anything >1800 seconds. > > With this split, quick could be used as part of any development, to get an > indication that everything is ok. For a sensible coverage before posting > any patch, quick and medium should do the job. The cronjobs we have running > at the moment could thus do 'quick,medium' for the daily one, and > 'quick,medium,long’ for the weekly one. > > Thoughts? Ideas? Additional comments? > > Thanks, > > Andreas > > > -- IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or store or copy the > information in any medium. Thank you. > > ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, > Registered in England & Wales, Company No: 2557590 > ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, > Registered in England & Wales, Company No: 2548782 > _______________________________________________ > gem5-dev mailing list > gem5-dev@gem5.org > http://m5sim.org/mailman/listinfo/gem5-dev > _______________________________________________ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
