My point is that you -don't- have it without the reference outputs. The reference outputs are how you make it exist in this case.
Gabe On 02/05/11 10:00, Steve Reinhardt wrote: > So what's the point in having it without reference outputs? Sure you > could run it, but as a regression it wouldn't be very valuable since > you wouldn't know if anything changed or not. I guess you could run > it to see if it completed, but I'm not sure how much value is in that > (or whether anyone would ever do it). > > I think some of the existing long regressions are already too long, > and I don't really want to see us add regressions that are any longer. > The goal of regressions is to make sure you didn't break anything, so > you want to maximize the variety of things you test. It's not clear > that (once you get above some threshold) running a test for longer is > going to exercise new code interactions that might actually expose > bugs... as Ali said, it's not clear that running 20 hours of bzip2 is > a more stressful test than 10 or 5 or even 2 hours of running bzip2 > (unless you're looking for a slow memory leak, but there are better > ways of catching that). > > Steve > > On Fri, Feb 4, 2011 at 11:58 PM, Gabe Black <[email protected] > <mailto:[email protected]>> wrote: > > That's actually what we're talking about. All the script bits are > already there, I'd just be adding reference outputs. > > Gabe > > > On 02/04/11 23:49, nathan binkert wrote: >> Can't he commit the regression without any reference stats? I >> thought it wouldn't run if there were no refs. Perhaps the >> script is so simple as to be not worth it? >> >> Nate >> >> On Fri, Feb 4, 2011 at 9:17 AM, Ali Saidi <[email protected] >> <mailto:[email protected]>> wrote: >> >> My opinion is that I've never had bzip find a bug that some >> other spec test hasn't. I think 20 hours is way too long for >> a regression test. We should either shrink the input size or >> just not have it. >> >> >> >> Ali >> >> >> >> On Thu, 03 Feb 2011 15:35:29 -0800, Gabe Black wrote: >> >> So how do people feel about this? bzip2 in x86 on O3 >> takes more than 20 hours to run. Should that be part of >> the regressions or not? I have it ready to go if that's >> acceptable, but to me it seems too long. >> >> Gabe >> >> On 02/01/11 17:30, Gabe Black wrote: >> >> Normally I wouldn't send stats updates out for >> review. I'm adding new regressions here, though, and >> I think people may be interested in the run time, >> especially for bzip2. I think it was about 20+ hours >> on opt on a big machine from HP. This may be trimmed >> down if/when performance bottlenecks get fixed since >> O3 will have to do less work, but cutting it in half >> is still a long, long time for a regression. I notice >> that the number of ticks for the benchmarks that are >> in common with Alpha is about 4 times higher, >> suggesting a performance bug of some sort (I doubt >> x86 gcc is 4X worse). >> >> Gabe >> >> On 02/01/11 17:23, Gabe Black wrote: >> >> This is an automatically generated e-mail. To >> reply, visit: http://reviews.m5sim.org/r/463/ >> >> >> Review request for Default, Ali Saidi, Gabe >> Black, Steve Reinhardt, and Nathan Binkert. >> By Gabe Black. >> >> >> Description >> >> X86: Add o3 regressions in SE mode. >> >> >> Diffs >> >> * >> tests/long/00.gzip/ref/x86/linux/o3-timing/config.ini >> (PRE-CREATION) >> * tests/long/00.gzip/ref/x86/linux/o3-timing/simerr >> (PRE-CREATION) >> * tests/long/00.gzip/ref/x86/linux/o3-timing/simout >> (PRE-CREATION) >> * >> tests/long/00.gzip/ref/x86/linux/o3-timing/stats.txt >> (PRE-CREATION) >> * >> tests/long/10.mcf/ref/x86/linux/o3-timing/config.ini >> (PRE-CREATION) >> * tests/long/10.mcf/ref/x86/linux/o3-timing/mcf.out >> (PRE-CREATION) >> * tests/long/10.mcf/ref/x86/linux/o3-timing/simerr >> (PRE-CREATION) >> * tests/long/10.mcf/ref/x86/linux/o3-timing/simout >> (PRE-CREATION) >> * tests/long/10.mcf/ref/x86/linux/o3-timing/stats.txt >> (PRE-CREATION) >> * >> tests/long/20.parser/ref/x86/linux/o3-timing/config.ini >> (PRE-CREATION) >> * tests/long/20.parser/ref/x86/linux/o3-timing/simerr >> (PRE-CREATION) >> * tests/long/20.parser/ref/x86/linux/o3-timing/simout >> (PRE-CREATION) >> * >> tests/long/20.parser/ref/x86/linux/o3-timing/stats.txt >> (PRE-CREATION) >> * >> tests/long/60.bzip2/ref/x86/linux/o3-timing/config.ini >> (PRE-CREATION) >> * tests/long/60.bzip2/ref/x86/linux/o3-timing/simerr >> (PRE-CREATION) >> * tests/long/60.bzip2/ref/x86/linux/o3-timing/simout >> (PRE-CREATION) >> * >> tests/long/60.bzip2/ref/x86/linux/o3-timing/stats.txt >> (PRE-CREATION) >> * >> tests/long/70.twolf/ref/x86/linux/o3-timing/config.ini >> (PRE-CREATION) >> * tests/long/70.twolf/ref/x86/linux/o3-timing/simerr >> (PRE-CREATION) >> * tests/long/70.twolf/ref/x86/linux/o3-timing/simout >> (PRE-CREATION) >> * >> tests/long/70.twolf/ref/x86/linux/o3-timing/smred.out >> (PRE-CREATION) >> * >> tests/long/70.twolf/ref/x86/linux/o3-timing/smred.pin >> (PRE-CREATION) >> * >> tests/long/70.twolf/ref/x86/linux/o3-timing/smred.pl1 >> (PRE-CREATION) >> * >> tests/long/70.twolf/ref/x86/linux/o3-timing/smred.pl2 >> (PRE-CREATION) >> * >> tests/long/70.twolf/ref/x86/linux/o3-timing/smred.sav >> (PRE-CREATION) >> * >> tests/long/70.twolf/ref/x86/linux/o3-timing/smred.sv2 >> (PRE-CREATION) >> * >> tests/long/70.twolf/ref/x86/linux/o3-timing/smred.twf >> (PRE-CREATION) >> * >> tests/long/70.twolf/ref/x86/linux/o3-timing/stats.txt >> (PRE-CREATION) >> * >> tests/quick/00.hello/ref/x86/linux/o3-timing/config.ini >> (PRE-CREATION) >> * tests/quick/00.hello/ref/x86/linux/o3-timing/simerr >> (PRE-CREATION) >> * tests/quick/00.hello/ref/x86/linux/o3-timing/simout >> (PRE-CREATION) >> * >> tests/quick/00.hello/ref/x86/linux/o3-timing/stats.txt >> (PRE-CREATION) >> >> View Diff <http://reviews.m5sim.org/r/463/diff/> >> >> _______________________________________________ >> m5-dev mailing list [email protected] >> <mailto:[email protected]> >> http://m5sim.org/mailman/listinfo/m5-dev >> >> >> _______________________________________________ >> m5-dev mailing list [email protected] >> <mailto:[email protected]> >> http://m5sim.org/mailman/listinfo/m5-dev >> >> >> >> >> >> _______________________________________________ >> m5-dev mailing list >> [email protected] <mailto:[email protected]> >> http://m5sim.org/mailman/listinfo/m5-dev >> >> >> >> _______________________________________________ >> m5-dev mailing list >> [email protected] <mailto:[email protected]> >> http://m5sim.org/mailman/listinfo/m5-dev > > > _______________________________________________ > m5-dev mailing list > [email protected] <mailto:[email protected]> > http://m5sim.org/mailman/listinfo/m5-dev > > > > _______________________________________________ > m5-dev mailing list > [email protected] > http://m5sim.org/mailman/listinfo/m5-dev
_______________________________________________ m5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/m5-dev
