Re: [m5-dev] m5-stable: 11 new changesets
nathan binkert wrote: We should probably open this discussion up to the m5-users list a bit to get some feedback and see how many people even plan to track us. We're likely to want to sync with a stable, well tested, m5 state a few times a year. Having an m5 stability point every 6 months is less frequent than I'd prefer. Forcing yourselves to do it every month, leaving only 50% of the time available for general commits, is probably more than we (HAsim) need from the repository. If less frequent intervals works for others then requiring half a month of stabilization every other month would reduce the lock-out to only 25% of the time. -Michael smime.p7s Description: S/MIME Cryptographic Signature ___ m5-dev mailing list m5-dev@m5sim.org http://m5sim.org/mailman/listinfo/m5-dev
Re: [m5-dev] m5-stable: 11 new changesets
> I figured we'd make an exception for this month until all the pending > changes get in. We can start the new schedule in July (so the > earliest m5-stable update under the new plan would be ~Aug 1). Should > we should freeze m5-stable where it is now, or if not, when? Ok, I like that. I think stable is alright since we've gotten a bunch of important fixes in and not much has happened in a while. That said, not too many users have grabbed it, so it might be too soon to tell. We should probably open this discussion up to the m5-users list a bit to get some feedback and see how many people even plan to track us. Nate ___ m5-dev mailing list m5-dev@m5sim.org http://m5sim.org/mailman/listinfo/m5-dev
Re: [m5-dev] m5-stable: 11 new changesets
On Sun, Jun 15, 2008 at 8:55 PM, nathan binkert <[EMAIL PROTECTED]> wrote: > So when do we start? Are we now into week three (only bugfix type > changes)? I personally would like a few more days to get in some of > the changes I've had in my tree for ages. I figured we'd make an exception for this month until all the pending changes get in. We can start the new schedule in July (so the earliest m5-stable update under the new plan would be ~Aug 1). Should we should freeze m5-stable where it is now, or if not, when? Steve ___ m5-dev mailing list m5-dev@m5sim.org http://m5sim.org/mailman/listinfo/m5-dev
Re: [m5-dev] m5-stable: 11 new changesets
>>> In the case of some bug fix that needs to be released immediately that >>> fix could be pushed directly to m5-stable and pulled into m5. >> This is an intriguing idea. OpenBSD does this on a 6 month schedule >> and it works pretty well. Several other groups (ubuntu for example) >> have also started on doing this. 6 months would be way too much for >> us, but I think on this shorter schedule, it might be doable. > > Yea, I like this too. So when do we start? Are we now into week three (only bugfix type changes)? I personally would like a few more days to get in some of the changes I've had in my tree for ages. Nate ___ m5-dev mailing list m5-dev@m5sim.org http://m5sim.org/mailman/listinfo/m5-dev
Re: [m5-dev] m5-stable: 11 new changesets
>> It >> seems that certain things (like x86) ought to be exempt from this rule >> though. > > Why? I'd rather establish exceptions when the need becomes apparent > rather than up front. I guess I figured that we could avoid hanging gabe up, but I guess he can always commit to his local repository and just push when the repo is open. Nate ___ m5-dev mailing list m5-dev@m5sim.org http://m5sim.org/mailman/listinfo/m5-dev
Re: [m5-dev] m5-stable: 11 new changesets
On Sun, Jun 15, 2008 at 8:19 PM, nathan binkert <[EMAIL PROTECTED]> wrote: >> What about something along the lines of the following: The m5-stable >> repository gets updated at a regular interval (1 month works for an >> example and it seems like a reasonable time frame to me). Instead of >> coming up with some conditions when m5-stable is updated from m5 >> automagically, we also impose a schedule on committing to the m5 >> repository. For example for the first 15 days of each month, any >> patches that pass regressions and the committer has tested and >> believes work can be committed. For the next week only minor changes >> can be committed, and finally for the forth week of each month only >> bugfixes can be committed. On the last day of the month the repository >> should be rather stable. Any largish changes would have been tested >> for 2 weeks. >> >> In the case of some bug fix that needs to be released immediately that >> fix could be pushed directly to m5-stable and pulled into m5. > This is an intriguing idea. OpenBSD does this on a 6 month schedule > and it works pretty well. Several other groups (ubuntu for example) > have also started on doing this. 6 months would be way too much for > us, but I think on this shorter schedule, it might be doable. Yea, I like this too. > It > seems that certain things (like x86) ought to be exempt from this rule > though. Why? I'd rather establish exceptions when the need becomes apparent rather than up front. > > The big issue is collaboration among people (like me and steve) and > how we need to do that, but I guess with mq and local repositories, > we've got enough tools. I'm sure we could work it out as needed. Steve ___ m5-dev mailing list m5-dev@m5sim.org http://m5sim.org/mailman/listinfo/m5-dev
Re: [m5-dev] m5-stable: 11 new changesets
> What about something along the lines of the following: The m5-stable > repository gets updated at a regular interval (1 month works for an > example and it seems like a reasonable time frame to me). Instead of > coming up with some conditions when m5-stable is updated from m5 > automagically, we also impose a schedule on committing to the m5 > repository. For example for the first 15 days of each month, any > patches that pass regressions and the committer has tested and > believes work can be committed. For the next week only minor changes > can be committed, and finally for the forth week of each month only > bugfixes can be committed. On the last day of the month the repository > should be rather stable. Any largish changes would have been tested > for 2 weeks. > > In the case of some bug fix that needs to be released immediately that > fix could be pushed directly to m5-stable and pulled into m5. This is an intriguing idea. OpenBSD does this on a 6 month schedule and it works pretty well. Several other groups (ubuntu for example) have also started on doing this. 6 months would be way too much for us, but I think on this shorter schedule, it might be doable. It seems that certain things (like x86) ought to be exempt from this rule though. The big issue is collaboration among people (like me and steve) and how we need to do that, but I guess with mq and local repositories, we've got enough tools. Nate ___ m5-dev mailing list m5-dev@m5sim.org http://m5sim.org/mailman/listinfo/m5-dev
Re: [m5-dev] m5-stable: 11 new changesets
On Jun 15, 2008, at 10:47 PM, nathan binkert wrote: > >> Maybe the only practical approach to improving the situation is to >> beef up the regressions. Other than the OS X bits, we could probably >> get a lot of mileage out of setting up a bunch of VMs on zizzer. We >> should also be more proactive about creating new regressions whenever >> someone turns up a bug that the current regressions didn't catch. > I am in favor of this approach. Is it even necessary to create many > VMs, or is it enough to just have me compile many different versions > of things and try many combinations? As for OS X, I think that > between Ali and I, we can keep things going. I can let my machine do > a regression once a week or something. Actually, the compiling is the > important part. It's pretty rare that for m5 to compile correctly on > osx and on linux and run successfully on linux but fail to run on osx. With some things different versions interact reasonably well (python and gcc come to mind) however, I think swig probably doesn't. I'm really less worried about a problem M5 not compiling for some combination of build tools (these are very easy to fix and clear when they happen), I'm more worried about subtle bugs getting introduced that our regressions aren't catching or the breaking of features that others are using. What about something along the lines of the following: The m5-stable repository gets updated at a regular interval (1 month works for an example and it seems like a reasonable time frame to me). Instead of coming up with some conditions when m5-stable is updated from m5 automagically, we also impose a schedule on committing to the m5 repository. For example for the first 15 days of each month, any patches that pass regressions and the committer has tested and believes work can be committed. For the next week only minor changes can be committed, and finally for the forth week of each month only bugfixes can be committed. On the last day of the month the repository should be rather stable. Any largish changes would have been tested for 2 weeks. In the case of some bug fix that needs to be released immediately that fix could be pushed directly to m5-stable and pulled into m5. Thoughts? Ali ___ m5-dev mailing list m5-dev@m5sim.org http://m5sim.org/mailman/listinfo/m5-dev
Re: [m5-dev] m5-stable: 11 new changesets
> I think the bottom line is that as long as you want to do extensive > non-automated testing before declaring something as "stable" then you > have to have (1) some sort of freeze prior to that point so that your > testers are all testing the same thing (modulo bug fixes) and (2) some > group of people who are going to be doing that testing, i.e., neither > waiting for "stable" nor trying to stay on the bleeding edge by > bypassing the freeze. Maybe it's not realistic to expect that here. I agree. > Maybe the only practical approach to improving the situation is to > beef up the regressions. Other than the OS X bits, we could probably > get a lot of mileage out of setting up a bunch of VMs on zizzer. We > should also be more proactive about creating new regressions whenever > someone turns up a bug that the current regressions didn't catch. I am in favor of this approach. Is it even necessary to create many VMs, or is it enough to just have me compile many different versions of things and try many combinations? As for OS X, I think that between Ali and I, we can keep things going. I can let my machine do a regression once a week or something. Actually, the compiling is the important part. It's pretty rare that for m5 to compile correctly on osx and on linux and run successfully on linux but fail to run on osx. Nate ___ m5-dev mailing list m5-dev@m5sim.org http://m5sim.org/mailman/listinfo/m5-dev
Re: [m5-dev] m5-stable: 11 new changesets
I sympathize with Ali's concerns... the current situation is pretty much the opposite extreme of the 3-6 month release cycle I was advocating not that long ago. My main motivation was that a longer cycle would allow more time for people to find problems, and the criticism was that a lot of the problems were being found by people who would be pulling from m5-stable anyway, so the latency didn't help. But now we're at the other end of the spectrum, where even the people who pull from the dev repo aren't getting a chance to try it. The auto-update after regression strategy is nice in theory, but as long as our regressions are as feeble as they currently are then Ali's got a point. And I'm not talking about just the number of tests, but also running the tests on a variety of platforms (linux, OS X, 32-bit, 64-bit, etc.) and with a variety of versions of gcc, scons, swig, python, etc. I don't have a great solution other than beefing up the regressions... imposing a minimum age would be awkward, as that implies that we'd run the regressions on a snapshot that's a week old since that would be the "release candidate", but of course you want to be testing the head of the tree as well. There's also the question of how you'd get urgent patches into m5-stable if there's a significant latency yet we want m5-stable to simply track the dev repo. (I guess we have that problem already, but increasing the latency would just make it worse.) Thinking about that gets you back to the idea of keeping the dev repo as a patch queue on top of m5-stable, but we already rejected that as too ugly (and probably rightly so). I think the bottom line is that as long as you want to do extensive non-automated testing before declaring something as "stable" then you have to have (1) some sort of freeze prior to that point so that your testers are all testing the same thing (modulo bug fixes) and (2) some group of people who are going to be doing that testing, i.e., neither waiting for "stable" nor trying to stay on the bleeding edge by bypassing the freeze. Maybe it's not realistic to expect that here. Maybe the only practical approach to improving the situation is to beef up the regressions. Other than the OS X bits, we could probably get a lot of mileage out of setting up a bunch of VMs on zizzer. We should also be more proactive about creating new regressions whenever someone turns up a bug that the current regressions didn't catch. I could be wrong though... Steve ___ m5-dev mailing list m5-dev@m5sim.org http://m5sim.org/mailman/listinfo/m5-dev
Re: [m5-dev] m5-stable: 11 new changesets
On Jun 15, 2008, at 12:12 PM, nathan binkert wrote: >> I think we might want to amend the policy to all regression tests >> passing and 1 week going by or something. I haven't even pulled the >> repository and tested a single thing yet and the regressions as they >> currently stand don't cover everything. > > My concern with a 1 week policy is that who is going to do the update > then? How would this work? we commit last week's successful > regression before we run this week's? How do we bail out? I'll work > on making the regression framework better so we can get more coverage > so it is more reliable. I don't know exactly how it should work, hence the discussion. Perhaps it's the job of the committer to pull their changeset into the stable repository a week after it's been committed. It would be rather easy to write a script that e-mailed the committer reminding them. > I did compile all targets on several machines and ran regressions. > All of my code has also been tested by me and others at HP for months. Yea, my concern isn't this particular change set, but the policy in general. I don't think that committing code and midnight, having regressions run and 3 am, and the that code getting pushed to the "stable" repository at 8 am is going to keep the stable repository stable. It seems like there must be some amount of use by people other than the committer to verify that a change didn't break a feature that the committer doesn't use, but others may. Ali ___ m5-dev mailing list m5-dev@m5sim.org http://m5sim.org/mailman/listinfo/m5-dev
Re: [m5-dev] m5-stable: 11 new changesets
> I think we might want to amend the policy to all regression tests > passing and 1 week going by or something. I haven't even pulled the > repository and tested a single thing yet and the regressions as they > currently stand don't cover everything. My concern with a 1 week policy is that who is going to do the update then? How would this work? we commit last week's successful regression before we run this week's? How do we bail out? I'll work on making the regression framework better so we can get more coverage so it is more reliable. I did compile all targets on several machines and ran regressions. All of my code has also been tested by me and others at HP for months. Nate ___ m5-dev mailing list m5-dev@m5sim.org http://m5sim.org/mailman/listinfo/m5-dev
Re: [m5-dev] m5-stable: 11 new changesets
I think we might want to amend the policy to all regression tests
passing and 1 week going by or something. I haven't even pulled the
repository and tested a single thing yet and the regressions as they
currently stand don't cover everything.
Ali
On Jun 15, 2008, at 10:51 AM, Nathan Binkert wrote:
> I manually pulled stable since all regressions passed.
>
> Nate
>
> On Sun, Jun 15, 2008 at 7:46 AM, Nathan Binkert <[EMAIL PROTECTED]>
> wrote:
>> changeset ca74f6845b27 in /z/repo/m5-stable
>> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=ca74f6845b27
>> summary: Add missing dependencies on .i files
>>
>> changeset 7eb7f0f5e79f in /z/repo/m5-stable
>> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=7eb7f0f5e79f
>> summary: Fix various SWIG warnings
>>
>> changeset 4cff095bbf2b in /z/repo/m5-stable
>> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=4cff095bbf2b
>> summary: Add hg commands for style check so you can check at times
>> other than commit
>>
>> changeset a1981d557252 in /z/repo/m5-stable
>> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=a1981d557252
>> summary: MemReq: Add option to reset the time on a request.
>>
>> changeset 6d9df90d70d7 in /z/repo/m5-stable
>> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=6d9df90d70d7
>> summary: python: Move various utility classes into a new m5.util
>> package so
>>
>> changeset 786868ff3058 in /z/repo/m5-stable
>> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=786868ff3058
>> summary: params: Fix floating point parameters
>>
>> changeset 42719798884a in /z/repo/m5-stable
>> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=42719798884a
>> summary: params: Fix the memory bandwidth parameter
>>
>> changeset ad060d1f1037 in /z/repo/m5-stable
>> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=ad060d1f1037
>> summary: python: Separate the options parsing stuff. Remove
>> options parsing stuff from
>>
>> changeset 576aa675d4e5 in /z/repo/m5-stable
>> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=576aa675d4e5
>> summary: Add .m5 configuration directory
>>
>> changeset 5df361e08b81 in /z/repo/m5-stable
>> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=5df361e08b81
>> summary: main: add .m5/options.py processing. This file is
>> processed before
>>
>> changeset 47c5168d092c in /z/repo/m5-stable
>> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=47c5168d092c
>> summary: Command line option to print out List of SimObjects and
>> their parameters
>>
>> diffstat:
>>
>> 22 files changed, 554 insertions(+), 735 deletions(-)
>> src/arch/mips/MipsTLB.py|1
>> src/arch/x86/X86TLB.py |1
>> src/python/SConscript |4
>> src/python/m5/attrdict.py | 30
>> src/python/m5/config.py | 21 +++
>> src/python/m5/main.py |5
>> src/python/m5/multidict.py | 91 -
>> src/python/m5/options.py| 72 ++
>> src/python/m5/params.py |3
>> src/python/m5/util.py | 29
>> src/python/m5/util/__init__.py | 16 ++
>> src/python/m5/util/attrdict.py | 35 +
>> src/python/m5/util/jobfile.py | 225 +
>> +++
>> src/python/m5/util/misc.py | 43 ++
>> src/python/m5/util/multidict.py | 91 +
>> src/python/m5/util/orderdict.py | 40 +
>> src/python/swig/event.i |1
>> src/python/swig/range.i |1
>> util/batch/jobfile.py | 269
>> ---
>> util/pbs/jobfile.py | 269
>> ---
>> util/stats/orderdict.py | 40 -
>> util/style.py |2
>>
>> diffs (truncated from 3101 to 300 lines):
>>
>> diff -r 52960521706f -r 47c5168d092c src/SConscript
>> --- a/src/SConscriptSat Jun 14 10:30:18 2008 -0700
>> +++ b/src/SConscriptSat Jun 14 21:51:08 2008 -0700
>> @@ -266,6 +266,7 @@ for name,simobj in generate.sim_objects.
>>env.Depends(hh_file, depends + extra_deps)
>>
>> # Generate any parameter header files needed
>> +params_i_files = []
>> for name,param in generate.params.iteritems():
>>if isinstance(param, m5.params.VectorParamDesc):
>>ext = 'vptype'
>> @@ -273,6 +274,7 @@ for name,param in generate.params.iterit
>>ext = 'ptype'
>>
>>i_file = File('params/%s_%s.i' % (name, ext))
>> +params_i_files.append(i_file)
>>env.Command(i_file, Value(name), generate.createSwigParam)
>>env.Depends(i_file, depends)
>>
>> @@ -295,7 +297,7 @@ names = sort_list(generate.sim_objects.k
>> names = sort_list(generate.sim_objects.keys())
>> env.Command(params_file, [ Value(v) for v in names ],
>>generate.buildParams)
>> -env.Depends(params_file, params_hh_files + depends)
>> +env.Depends(params_file, params_hh_files + params_i_files + depends)
>> SwigSource('m5.objects', params_file
Re: [m5-dev] m5-stable: 11 new changesets
I manually pulled stable since all regressions passed.
Nate
On Sun, Jun 15, 2008 at 7:46 AM, Nathan Binkert <[EMAIL PROTECTED]> wrote:
> changeset ca74f6845b27 in /z/repo/m5-stable
> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=ca74f6845b27
> summary: Add missing dependencies on .i files
>
> changeset 7eb7f0f5e79f in /z/repo/m5-stable
> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=7eb7f0f5e79f
> summary: Fix various SWIG warnings
>
> changeset 4cff095bbf2b in /z/repo/m5-stable
> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=4cff095bbf2b
> summary: Add hg commands for style check so you can check at times other than
> commit
>
> changeset a1981d557252 in /z/repo/m5-stable
> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=a1981d557252
> summary: MemReq: Add option to reset the time on a request.
>
> changeset 6d9df90d70d7 in /z/repo/m5-stable
> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=6d9df90d70d7
> summary: python: Move various utility classes into a new m5.util package so
>
> changeset 786868ff3058 in /z/repo/m5-stable
> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=786868ff3058
> summary: params: Fix floating point parameters
>
> changeset 42719798884a in /z/repo/m5-stable
> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=42719798884a
> summary: params: Fix the memory bandwidth parameter
>
> changeset ad060d1f1037 in /z/repo/m5-stable
> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=ad060d1f1037
> summary: python: Separate the options parsing stuff. Remove options parsing
> stuff from
>
> changeset 576aa675d4e5 in /z/repo/m5-stable
> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=576aa675d4e5
> summary: Add .m5 configuration directory
>
> changeset 5df361e08b81 in /z/repo/m5-stable
> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=5df361e08b81
> summary: main: add .m5/options.py processing. This file is processed before
>
> changeset 47c5168d092c in /z/repo/m5-stable
> details: http://repo.m5sim.org/m5-stable?cmd=changeset;node=47c5168d092c
> summary: Command line option to print out List of SimObjects and their
> parameters
>
> diffstat:
>
> 22 files changed, 554 insertions(+), 735 deletions(-)
> src/arch/mips/MipsTLB.py|1
> src/arch/x86/X86TLB.py |1
> src/python/SConscript |4
> src/python/m5/attrdict.py | 30
> src/python/m5/config.py | 21 +++
> src/python/m5/main.py |5
> src/python/m5/multidict.py | 91 -
> src/python/m5/options.py| 72 ++
> src/python/m5/params.py |3
> src/python/m5/util.py | 29
> src/python/m5/util/__init__.py | 16 ++
> src/python/m5/util/attrdict.py | 35 +
> src/python/m5/util/jobfile.py | 225
> src/python/m5/util/misc.py | 43 ++
> src/python/m5/util/multidict.py | 91 +
> src/python/m5/util/orderdict.py | 40 +
> src/python/swig/event.i |1
> src/python/swig/range.i |1
> util/batch/jobfile.py | 269 ---
> util/pbs/jobfile.py | 269 ---
> util/stats/orderdict.py | 40 -
> util/style.py |2
>
> diffs (truncated from 3101 to 300 lines):
>
> diff -r 52960521706f -r 47c5168d092c src/SConscript
> --- a/src/SConscriptSat Jun 14 10:30:18 2008 -0700
> +++ b/src/SConscriptSat Jun 14 21:51:08 2008 -0700
> @@ -266,6 +266,7 @@ for name,simobj in generate.sim_objects.
> env.Depends(hh_file, depends + extra_deps)
>
> # Generate any parameter header files needed
> +params_i_files = []
> for name,param in generate.params.iteritems():
> if isinstance(param, m5.params.VectorParamDesc):
> ext = 'vptype'
> @@ -273,6 +274,7 @@ for name,param in generate.params.iterit
> ext = 'ptype'
>
> i_file = File('params/%s_%s.i' % (name, ext))
> +params_i_files.append(i_file)
> env.Command(i_file, Value(name), generate.createSwigParam)
> env.Depends(i_file, depends)
>
> @@ -295,7 +297,7 @@ names = sort_list(generate.sim_objects.k
> names = sort_list(generate.sim_objects.keys())
> env.Command(params_file, [ Value(v) for v in names ],
> generate.buildParams)
> -env.Depends(params_file, params_hh_files + depends)
> +env.Depends(params_file, params_hh_files + params_i_files + depends)
> SwigSource('m5.objects', params_file)
>
> # Build all swig modules
> diff -r 52960521706f -r 47c5168d092c src/arch/mips/MipsTLB.py
> --- a/src/arch/mips/MipsTLB.py Sat Jun 14 10:30:18 2008 -0700
> +++ b/src/arch/mips/MipsTLB.py Sat Jun 14 21:51:08 2008 -0700
> @@ -33,10 +33,8 @@ from m5.params import *
> from m5.params import *
>
> class MipsTLB(SimObject):
> +type = 'MipsTLB'
> abstract = True
> -type = 'MipsTLB'
> -cxx_namespace = 'MipsISA'
> -
