All true. I guess I'm thinking of the regressions as an example of how to run a job, but I really shouldn't since that's not what they're for.
Nate On Sat, Feb 5, 2011 at 10:00 AM, Steve Reinhardt <[email protected]> wrote: > So what's the point in having it without reference outputs? Sure you could > run it, but as a regression it wouldn't be very valuable since you wouldn't > know if anything changed or not. I guess you could run it to see if it > completed, but I'm not sure how much value is in that (or whether anyone > would ever do it). > > I think some of the existing long regressions are already too long, and I > don't really want to see us add regressions that are any longer. The goal > of regressions is to make sure you didn't break anything, so you want to > maximize the variety of things you test. It's not clear that (once you get > above some threshold) running a test for longer is going to exercise new > code interactions that might actually expose bugs... as Ali said, it's not > clear that running 20 hours of bzip2 is a more stressful test than 10 or 5 > or even 2 hours of running bzip2 (unless you're looking for a slow memory > leak, but there are better ways of catching that). > > Steve > > On Fri, Feb 4, 2011 at 11:58 PM, Gabe Black <[email protected]> wrote: > >> That's actually what we're talking about. All the script bits are already >> there, I'd just be adding reference outputs. >> >> Gabe >> >> >> On 02/04/11 23:49, nathan binkert wrote: >> >> Can't he commit the regression without any reference stats? I thought it >> wouldn't run if there were no refs. Perhaps the script is so simple as to >> be not worth it? >> >> Nate >> >> On Fri, Feb 4, 2011 at 9:17 AM, Ali Saidi <[email protected]> wrote: >> >>> My opinion is that I've never had bzip find a bug that some other spec >>> test hasn't. I think 20 hours is way too long for a regression test. We >>> should either shrink the input size or just not have it. >>> >>> >>> >>> Ali >>> >>> >>> >>> On Thu, 03 Feb 2011 15:35:29 -0800, Gabe Black wrote: >>> >>> So how do people feel about this? bzip2 in x86 on O3 takes more than 20 >>> hours to run. Should that be part of the regressions or not? I have it ready >>> to go if that's acceptable, but to me it seems too long. >>> >>> Gabe >>> >>> On 02/01/11 17:30, Gabe Black wrote: >>> >>> Normally I wouldn't send stats updates out for review. I'm adding new >>> regressions here, though, and I think people may be interested in the run >>> time, especially for bzip2. I think it was about 20+ hours on opt on a big >>> machine from HP. This may be trimmed down if/when performance bottlenecks >>> get fixed since O3 will have to do less work, but cutting it in half is >>> still a long, long time for a regression. I notice that the number of ticks >>> for the benchmarks that are in common with Alpha is about 4 times higher, >>> suggesting a performance bug of some sort (I doubt x86 gcc is 4X worse). >>> >>> Gabe >>> >>> On 02/01/11 17:23, Gabe Black wrote: >>> >>> This is an automatically generated e-mail. To reply, visit: >>> http://reviews.m5sim.org/r/463/ >>> Review request for Default, Ali Saidi, Gabe Black, Steve Reinhardt, >>> and Nathan Binkert. >>> By Gabe Black. >>> Description >>> >>> X86: Add o3 regressions in SE mode. >>> >>> Diffs >>> >>> - tests/long/00.gzip/ref/x86/linux/o3-timing/config.ini >>> (PRE-CREATION) >>> - tests/long/00.gzip/ref/x86/linux/o3-timing/simerr (PRE-CREATION) >>> - tests/long/00.gzip/ref/x86/linux/o3-timing/simout (PRE-CREATION) >>> - tests/long/00.gzip/ref/x86/linux/o3-timing/stats.txt (PRE-CREATION) >>> - tests/long/10.mcf/ref/x86/linux/o3-timing/config.ini (PRE-CREATION) >>> - tests/long/10.mcf/ref/x86/linux/o3-timing/mcf.out (PRE-CREATION) >>> - tests/long/10.mcf/ref/x86/linux/o3-timing/simerr (PRE-CREATION) >>> - tests/long/10.mcf/ref/x86/linux/o3-timing/simout (PRE-CREATION) >>> - tests/long/10.mcf/ref/x86/linux/o3-timing/stats.txt (PRE-CREATION) >>> - tests/long/20.parser/ref/x86/linux/o3-timing/config.ini >>> (PRE-CREATION) >>> - tests/long/20.parser/ref/x86/linux/o3-timing/simerr (PRE-CREATION) >>> - tests/long/20.parser/ref/x86/linux/o3-timing/simout (PRE-CREATION) >>> - tests/long/20.parser/ref/x86/linux/o3-timing/stats.txt >>> (PRE-CREATION) >>> - tests/long/60.bzip2/ref/x86/linux/o3-timing/config.ini >>> (PRE-CREATION) >>> - tests/long/60.bzip2/ref/x86/linux/o3-timing/simerr (PRE-CREATION) >>> - tests/long/60.bzip2/ref/x86/linux/o3-timing/simout (PRE-CREATION) >>> - tests/long/60.bzip2/ref/x86/linux/o3-timing/stats.txt >>> (PRE-CREATION) >>> - tests/long/70.twolf/ref/x86/linux/o3-timing/config.ini >>> (PRE-CREATION) >>> - tests/long/70.twolf/ref/x86/linux/o3-timing/simerr (PRE-CREATION) >>> - tests/long/70.twolf/ref/x86/linux/o3-timing/simout (PRE-CREATION) >>> - tests/long/70.twolf/ref/x86/linux/o3-timing/smred.out >>> (PRE-CREATION) >>> - tests/long/70.twolf/ref/x86/linux/o3-timing/smred.pin >>> (PRE-CREATION) >>> - tests/long/70.twolf/ref/x86/linux/o3-timing/smred.pl1 >>> (PRE-CREATION) >>> - tests/long/70.twolf/ref/x86/linux/o3-timing/smred.pl2 >>> (PRE-CREATION) >>> - tests/long/70.twolf/ref/x86/linux/o3-timing/smred.sav >>> (PRE-CREATION) >>> - tests/long/70.twolf/ref/x86/linux/o3-timing/smred.sv2 >>> (PRE-CREATION) >>> - tests/long/70.twolf/ref/x86/linux/o3-timing/smred.twf >>> (PRE-CREATION) >>> - tests/long/70.twolf/ref/x86/linux/o3-timing/stats.txt >>> (PRE-CREATION) >>> - tests/quick/00.hello/ref/x86/linux/o3-timing/config.ini >>> (PRE-CREATION) >>> - tests/quick/00.hello/ref/x86/linux/o3-timing/simerr (PRE-CREATION) >>> - tests/quick/00.hello/ref/x86/linux/o3-timing/simout (PRE-CREATION) >>> - tests/quick/00.hello/ref/x86/linux/o3-timing/stats.txt >>> (PRE-CREATION) >>> >>> View Diff <http://reviews.m5sim.org/r/463/diff/> >>> _______________________________________________ m5-dev mailing list >>> [email protected] http://m5sim.org/mailman/listinfo/m5-dev >>> >>> >>> _______________________________________________ m5-dev mailing list >>> [email protected] http://m5sim.org/mailman/listinfo/m5-dev >>> >>> >>> >>> >>> _______________________________________________ >>> m5-dev mailing list >>> [email protected] >>> http://m5sim.org/mailman/listinfo/m5-dev >>> >>> >> >> _______________________________________________ >> m5-dev mailing list >> [email protected]http://m5sim.org/mailman/listinfo/m5-dev >> >> >> >> _______________________________________________ >> >> m5-dev mailing list >> [email protected] >> http://m5sim.org/mailman/listinfo/m5-dev >> >> > > _______________________________________________ > m5-dev mailing list > [email protected] > http://m5sim.org/mailman/listinfo/m5-dev > >
_______________________________________________ m5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/m5-dev
