I'll see if I can put a little bit of effort into allowing multiple protocols to be compiled into the same binary. That would really fix the problem, right?
Nate On Mon, Feb 1, 2010 at 8:46 AM, Steve Reinhardt <ste...@gmail.com> wrote: > OK, I realize the idea I was originally thinking of didn't solve the > problem of not wanting to rerun all the non-ruby tests on every > different ruby protocol build. > > For those following at home (prob. everyone other than Brad and me), > the issue is that the test infrastructure is designed to run every > test that applies to a given binary, but with N different ruby > protocol builds, you end up running all of the ruby-independent tests > N times even though it should be sufficient to run them once. Brad's > solution only runs the ruby-independent tests on the build that uses > the "default" ruby protocol. However, that default protocol is > hard-coded in the test script, and when the actual default protocol > gets changed but the test script is not updated, then you end up not > running any ruby-independent tests (which is what has been happening > lately). > > Ideally I'd like to stay with the model of running every test on every > binary it applies to; while the current solution optimizes some > unnecessary tests out of the full regression, it also eliminates some > potentially useful tests for people who might not be using the default > ruby protocol. One option that I'm leaning toward is to add a scons > flag to disable compiling in the "classic" M5 cache model. Then we > can disable it in all but one of the ruby builds, and not run any > tests that require the classic M5 memory hierarchy for those builds. > That would still leave us running the purely functional tests (that > really don't require a cache hierarchy at all) on all of the different > ruby protocol builds, but I don't think any of those are very long, so > that shouldn't be so bad to put up with. > > Steve > > On Sun, Jan 31, 2010 at 10:53 PM, Steve Reinhardt <ste...@gmail.com> wrote: >> I'm not sure it's the stickiness... my interpretation is that Derek >> changed the default in the src/mem/protocol/SConsopts file which made >> it inconsistent with what you assumed it was in tests/SConscript. >> Anyway, I think the fix I suggested that completely removes any >> dependence on the default value in the test system is more robust and >> what we should do in the long run. >> >> Steve >> >> On Sun, Jan 31, 2010 at 10:33 PM, Beckmann, Brad <brad.beckm...@amd.com> >> wrote: >>> Yep, the problem is the PROTOCOL variable is sticky and therefore isn't >>> being set to the default. >>> >>> I just pushed a fix to add the default PROTOCOL variable to all build_opt >>> files. That should fix the regression tester, but it would be nice to >>> always set the PROTOCOL variable to the default unless explicitly specified >>> at the command line. >>> >>> Brad >>> >>> >>> -----Original Message----- >>> From: m5-dev-boun...@m5sim.org [mailto:m5-dev-boun...@m5sim.org] On Behalf >>> Of Steve Reinhardt >>> Sent: Sunday, January 31, 2010 4:40 PM >>> To: M5 Developer List >>> Subject: Re: [m5-dev] Cron <m5t...@zizzer> >>> /z/m5/regression/do-regression--scratch all >>> >>> OK, I believe it's a problem Brad was already worried about... see this: >>> >>> diff -r a162a2b0b1f8 -r 5bd33f7c26ea tests/SConscript >>> --- a/tests/SConscript Fri Jan 29 20:29:34 2010 -0800 >>> +++ b/tests/SConscript Fri Jan 29 20:29:40 2010 -0800 >>> @@ -264,11 +264,23 @@ >>> else: >>> configs += ['simple-atomic', 'simple-timing', 'o3-timing', 'memtest', >>> 'simple-atomic-mp', 'simple-timing-mp', 'o3-timing-mp', >>> - 'inorder-timing'] >>> + 'inorder-timing', 'rubytest'] >>> >>> if env['RUBY']: >>> - # Hack for Ruby >>> - configs += [c + '-ruby' for c in configs] >>> + # With Ruby, A protocol must be specified in the environment >>> + assert(env['PROTOCOL']) >>> + >>> + # >>> + # Is there a way to determine what is Protocol EnumVariable >>> + # default and eliminate the need to hard code the default protocol >>> below? >>> + # >>> + # If the binary includes the default ruby protocol, run both ruby and >>> + # non-ruby versions of the tests. Otherwise just run the ruby >>> versions. >>> + # >>> + if env['PROTOCOL'] == 'MI_example': >>> + configs += [c + "-ruby" for c in configs] >>> + else: >>> + configs = [c + "-ruby-" + env['PROTOCOL'] for c in configs] >>> >>> cwd = os.getcwd() >>> os.chdir(str(Dir('.').srcdir)) >>> >>> and then compare with this change, which is from Derek's merge changeset: >>> >>> diff -r 289ac904233d -r 3d308cbd1657 src/mem/protocol/SConsopts >>> --- a/src/mem/protocol/SConsopts Wed Nov 18 18:00:41 2009 -0800 >>> +++ b/src/mem/protocol/SConsopts Tue Jan 19 15:48:12 2010 -0600 >>> @@ -50,7 +50,7 @@ >>> 'MOESI_hammer', >>> ] >>> >>> -opt = EnumVariable('PROTOCOL', 'Coherence Protocol for Ruby', 'MI_example', >>> +opt = EnumVariable('PROTOCOL', 'Coherence Protocol for Ruby', >>> 'MOESI_CMP_directory', >>> all_protocols) >>> >>> >>> ... so that mismatch is causing all the builds that use the default >>> Ruby protocol to not find any applicable configurations since >>> tests/SConscript is looking for a different default protocol. >>> >>> I think the best way to fix this is to forget trying to have the test >>> configs track the 'default' Ruby protocol, but instead to always have >>> the protocol explicit in the config name. This will require moving >>> the existing *-ruby reference outputs, but it's more stable in the >>> long run. Brad and I can probably discuss this tomorrow and make it >>> happen. >>> >>> Steve >>> >>> 2010/1/31 Steve Reinhardt <ste...@gmail.com>: >>>> I'm looking at it now... I'm not sure exactly what the error is, but it >>>> looks like it's running *only* the ruby tests now, and acting like the >>>> non-ruby tests don't exist. >>>> >>>> Steve >>>> >>>> On Sun, Jan 31, 2010 at 11:24 AM, Beckmann, Brad <brad.beckm...@amd.com> >>>> wrote: >>>>> I agree, I don't think it is working correctly. I don't have access to >>>>> the Michigan server and I can't reproduce the error. Could someone send >>>>> out the do-regression script? >>>>> >>>>> Thanks, >>>>> >>>>> Brad >>> _______________________________________________ >>> m5-dev mailing list >>> m5-dev@m5sim.org >>> http://m5sim.org/mailman/listinfo/m5-dev >>> >>> >>> _______________________________________________ >>> m5-dev mailing list >>> m5-dev@m5sim.org >>> http://m5sim.org/mailman/listinfo/m5-dev >>> >> > _______________________________________________ > m5-dev mailing list > m5-dev@m5sim.org > http://m5sim.org/mailman/listinfo/m5-dev > > _______________________________________________ m5-dev mailing list m5-dev@m5sim.org http://m5sim.org/mailman/listinfo/m5-dev
