Hey Abhishek, Yes! Not only will we be releasing disk images (that we can depending on license for the benchmarks) and kernel images, we will also be releasing all of the documentation and the scripts describing how these images were created. These will also be updated at every release as appropriate.
Cheers, Jason On Thu, Dec 12, 2019 at 10:04 AM Abhishek Singh < abhishek.singh199...@gmail.com> wrote: > Hello Jason, > > This is perfect ! > > I had one more question, for full system simulations will there be release > of images and kernel files for every architectures (arm , x86, etc)? > > On Thu, Dec 12, 2019 at 12:57 PM Jason Lowe-Power <ja...@lowepower.com> > wrote: > > > Hey Abhishek, > > > > Emails will continue to gem5-dev on every changeset pushed to the develop > > branch as they do now to master :). We can discuss if we want the same > for > > feature branches (if we end up using them). > > > > Your first interpretation on gem5 stable is correct (sorry if this wasn't > > clear). It will be much more heavily tested than the minute-by-minute > > releases from the develop branch. With this testing we will be publishing > > the following: > > - What (common) workloads are supported (e.g., SPEC, parsec, etc.). Which > > workloads we use here will be discussed in the future, stay tuned. > > - For all of the workloads, we will publish common statistics for a few > > different systems (e.g., time to simulate, IPC, cache miss rates, memory > > bandwidth, etc). The systems used and the stats will be discussed in the > > future. > > > > The stable release will *not* be a continuous process. The only time the > > stable branch will be updated is 1) At releases or 2) if a "major" bug is > > encountered. For non-release updates (e.g., for bugs), we'll be very > > careful to either re-validate our results or somehow know the results > won't > > change. > > > > Cheers, > > Jason > > > > > > > > On Thu, Dec 12, 2019 at 9:38 AM Abhishek Singh < > > abhishek.singh199...@gmail.com> wrote: > > > > > Hi Jason, > > > > > > Thanks for your email. It cleared a lot of misunderstanding which I > had. > > > > > > Is it possible to have an email sent on every commit we make to atleast > > > gem5-dev? > > > The email list could be different and can be sent to people who are > > > interested in this so that it does not spam to gem5-Dev list. > > > > > > I am talking about gem5 developer branch and not stable. > > > > > > This is because, it will keep all the interested community members well > > > informed about new features that are added, who like to keep note of > > latest > > > changes and merge in their projects as required. > > > > > > And the way, I understand that a stable gem5 when a user desires is: > > > It wants the conference accepted applications (standard workload for > > > example spec, parsec, etc to run without any errors on both SE and FS > > mode > > > for every architecture and cpu models. > > > > > > If that’s what the stable releases are going to be testing before > > releasing > > > it, then a stable release is much more helpful and will have the wide > > > reach. But when I read proposal this seems to be step by step process > > > reaching it as the final goal. Please correct me on this if I > understood > > it > > > wrong. > > > > > > > > > The aim of stable releases is a continuous process and will always be > > > doubtful unless tested with all the major conference accepted > application > > > on every stable releases. > > > > > > On Thu, Dec 12, 2019 at 12:12 PM Jason Lowe-Power <ja...@lowepower.com > > > > > wrote: > > > > > > > Hi all, > > > > > > > > First of all, let me say that I hope it's clear that gem5 is not > > > > "controlled" in any way by me! As laid out in our governance > document ( > > > > http://gem5.org/Governance#Overview), "gem5 is a meritocratic, > > > > consensus-based community project". Through these emails, and as the > > > chair > > > > of the project management committee, I'm trying to *build* consensus. > > > > > > > > Importantly, there's a reason we're trying to make this push for > stable > > > > APIs. I've heard from dozens of current and potential gem5 users that > > > they > > > > want stable gem5 releases. By providing stable releases, we will be > > > > expanding the users of gem5 (and, IMO, making the research and papers > > > that > > > > use gem5 better). > > > > > > > > Could you please clarify the policy on breaking APIs? It makes sense > > for > > > > > releases to maintain stable APIs, but how does that apply to the > > > > > development branch? I'm worried that it will be very hard to make > > > changes > > > > > that don't change any interfaces, and we definitely don't want to > > > > encourage > > > > > a style of development where we just add and add and add without > ever > > > > > refactoring or consolidating things. If APIs can continue to change > > as > > > > > needed in the development branch and we just need to warn people > > before > > > > > they're released, that seems reasonable. > > > > > > > > > > > > Let's dig into this deeper. First, I'd like to remind everyone that > the > > > > current proposal was the community consensus reached in the gem5 > > roadmap > > > > document: > > > > > > > > > > > > > > https://docs.google.com/document/d/1fv01HavfkIIqfcgZoKUpojkUKkWujxspRCcvS5cTfkk/edit > > > > . > > > > Of course, we can always change what we decided before :). > > > > > > > > The goal is to provide our users with a stable base to build of off. > If > > > we > > > > are constantly changing interfaces, like we do today, it's impossible > > to > > > > build a project based off of gem5. There have been many projects that > > > have > > > > languished because of this including gem5+SST and dist-gem5. If we > had > > > well > > > > defined interfaces *that weren't constantly changing* I believe it > > would > > > > make gem5 a more widely used project. > > > > > > > > The proposal is to have *one* API breaking release every year. This > was > > > > decided based on community feedback in the roadmap document. > > > > > > > > I (personally) believe we need to slow down and be more deliberate > > about > > > > changing our interfaces. I think that it's a good thing that "it will > > be > > > > very hard to make changes that don't change any interfaces". This > will > > > make > > > > gem5 more stable and easy for others to use. If you believe you need > to > > > > change an interface, we should have a discussion about it first, and > > this > > > > should include a discussion about how the interface change will > affect > > > our > > > > users. > > > > > > > > I strongly agree that "we definitely don't want to encourage a style > of > > > > development where we just add and add and add without ever > refactoring > > or > > > > consolidating things." The current proposal states that when changing > > an > > > > interface, we first add a new function and mark the old version as > > > > deprecated. Then, during the API changing merge window delete all of > > the > > > > deprecated functions. This is going to take more maintenance from our > > > side, > > > > but we now have the resources to do this. > > > > > > > > I disagree that "we just need to warn people before [API changes] are > > > > released" is enough. We need to give them significant (months) of > lead > > > > time, and we need to give them the opportunity to transition. In > fact, > > I > > > > believe this is what Abhishek was getting at in his follow up. > > > > > > > > In general (and not necessarily tied to this proposal), we should try > > > very > > > > > hard not to have silos where people develop incompatible features > and > > > > don't > > > > > add to gem5 in a holistic way. It's easiest to wall off > functionality > > > > > behind ifdefs or config options or in separate repositories and > then > > > just > > > > > ignore it when implementing new features, but that creates a lot of > > > > > technical debt and complexity which really cripples both gem5 and > > > future > > > > > development and compounds exponentially over time. > > > > > > > > > > > > I agree with this statement. We need to encourage development in the > > open > > > > based on the mainline code. I strongly believe the current proposal > > > *makes > > > > this easier*, not harder. Today, many people do exactly what you're > > > > describing. By implementing stable APIs that people can build off of > > and > > > > allowing branches, we will be making it easier to add features to > gem5 > > > in a > > > > holistic way. > > > > > > > > Back to the branches, I'm still not a fan. I think it gives people an > > > > > excuse not to make changes ("you want me to refactor 18 months of > > code > > > > for > > > > > that?" (not an actual quote)), on top of other issues. Also there > are > > > > more > > > > > than two options. You can do your work on top of tree like I do, > even > > > > when > > > > > adding big new features. It ensures that you don't get too invested > > in > > > > bad > > > > > paths and don't have a mountain of integration work to do when you > > > > finally > > > > > decide to bridge back in. It also gives the community an > appropriate > > > > > opportunity to provide feedback through review which is not > practical > > > > when > > > > > dealing with dozens of files and thousands of lines of code with > > months > > > > of > > > > > yet unseen history and iteration. > > > > > > > > > > > > No one can do what you do, Gabe :D. > > > > > > > > More seriously, a couple of points: > > > > - Branches won't be "unseen." Their development and code review will > > > still > > > > happen on Gerrit. It just won't spam the list on every commit. > > > > - People already use this excuse not to make refactors. Or, (IMO much > > > > worse) they just give up and don't merge their changes. > > > > > > > > Lately, I've noticed we've been *very* slow to incorporate new > > features. > > > > Other than systemc and the fastmodel, the only major new feature in > the > > > > past couple of years I can think of is the ARM SVE changes, which > > > required > > > > a herculean effort and would have been much easier with branches, I > > > > believe. Some contributors have the resources to spend 100% of their > > time > > > > on gem5, but that's a rare case in industry and in academia. We have > to > > > > assume that most things will be developed by one or two people, over > > the > > > > course of years, spending less than 50% of their time on it. > > > > > > > > As a specific example, I know it would be easier for me and my > students > > > to > > > > upstream the flexcpu model we've created if we could just merge a > > branch > > > > (with the appropriate reviews, of course!). This has *required* a lot > > of > > > > out of mainline development. It's so far been almost two years and at > > > least > > > > three different students working on it to get it to a state where > it's > > > 90% > > > > working. It doesn't make sense to submit a patch with a half working > > CPU > > > > model. As it is, we've been spending a huge amount of time rebasing > and > > > > re-writing history to try to get the commits to make sense when we do > > > > finally push it. It would be easier if we could just incorporate > > changes > > > > from mainline by adding new changesets on top of what we've done > > instead > > > of > > > > having to rebase. This is also a good example of where I'd have loved > > to > > > > have a stable API. In the 1.5-2 years we've been working on it, the > > > > ExecContext interface has changed at least three times causing us to > > have > > > > to go through a major refactor each time. I'm certain we're not the > > only > > > > group going through this pain. > > > > > > > > Another use case is that most contributors want to wait until a paper > > is > > > > published before pushing their work. Developing in the open one > > changeset > > > > at a time just isn't the most common use case for gem5. > > > > > > > > I agree with Gabe, it seems like gem5 is taking a way to get better > > but > > > it > > > > > will very hard for PhD students to suddenly adapt and review > thousand > > > > lines > > > > > of code with months of yet unseen history and iteration and the > > reason > > > > > behind every change. > > > > > > > > > > > > Thanks for the feedback, Abhishek! > > > > > > > > Could you describe this in more detail? I'm not sure I understand > your > > > > concern. I'll go ahead and describe what I see as the new flow below, > > > > though. > > > > > > > > 1. On a minor release: > > > > You'll have a choice, you can either merge the new minor release into > > > your > > > > code or stay on the old release. The only reason you would want to > > merge > > > in > > > > the new minor release is that there is a new feature or a change in > > > > behavior that you want to take advantage of. Note: we will make it > very > > > > clear in the release notes what is changed each time. > > > > > > > > If you choose to merge in the new minor release, *it should be very > > > easy*. > > > > We will not break any interfaces, therefore the merge should be > simple. > > > As > > > > long as you only used "official" APIs, your code should work *without > > any > > > > changes*. > > > > > > > > 2. On a major release > > > > Again, you'll have the choice to merge or not. If there aren't any > new > > > > features you need, don't merge! If there are new features, then you > may > > > > want to take the effort to merge. > > > > > > > > To merge at a major release, you'll probably have to update your code > > > since > > > > APIs may have changed. These will be very well documented in the > > release > > > > notes with information on how to update your code. Additionally, > > assuming > > > > that you pulled the previous minor releases, you will have known this > > was > > > > coming because any APIs that have changed would have been marked as > > > > deprecated and produced compile time or run time warnings. > > > > > > > > 3. Contributing your code back to gem5 > > > > If you have a big new feature (e.g., a new CPU model), you can email > > > > gem5-dev and request a new branch. We'll set that up and then we can > > > start > > > > reviewing the code! There will likely be some refactoring that's > > required > > > > or merging current development changes. You can do this *on top* of > > your > > > > current changes, instead of having to rebase 50 changesets :). Once > the > > > > code is to a point that the maintainer and the community are happy, > > then > > > it > > > > will be a simple merge commit into the develop branch! > > > > > > > > Of course, small bugfixes should just be on top of develop. And, if > > your > > > > big new feature requires API changes, then it may be more effort to > get > > > the > > > > branch merged. > > > > > > > > If this doesn't make it clear, please let me know! > > > > > > > > ------------ > > > > > > > > To conclude: I'm trying to build consensus here. I want to see gem5 > > used > > > as > > > > broadly as possible; I want to see gem5 used to produce sustainable, > > > > reproducible, and scientifically rigorous research; and I want to see > > > gem5 > > > > be easy to use. I believe this new release model will help with these > > > > things, but I'm 100% open to modifications. > > > > > > > > I would appreciate specific actionable feedback. Simply saying "I > don't > > > > like the idea" is helpful, but it would be more helpful to provide > > > updates > > > > the proposal with your ideas on how to accomplish the goals of the > gem5 > > > > community project (again, see the Governance document: > > > > http://gem5.org/Governance#Philosophy). > > > > > > > > Cheers, > > > > Jason > > > > > > > > On Wed, Dec 11, 2019 at 7:49 PM Abhishek Singh < > > > > abhishek.singh199...@gmail.com> wrote: > > > > > > > > > > Hi Jason, > > > > > I agree with Gabe, it seems like gem5 is taking a way to get > better > > > but > > > > it > > > > > will very hard for PhD students to suddenly adapt and review > thousand > > > > lines > > > > > of code with months of yet unseen history and iteration and the > > reason > > > > > behind every change. > > > > > > > > > > And in the end it may end up to be used as a course project rather > > than > > > > > thesis project or project for architecture conferences as people > will > > > use > > > > > the commit or the branch which they are most familiar with. > > > > > With so less releases every year, personally it will take a lot of > > time > > > > to > > > > > relearn and understand the implementation in detail. > > > > > > > > > > But as this fully controlled by you, we can only suggest you. It > > should > > > > not > > > > > be like people are creating another public gem5 as we have now and > > use > > > > that. > > > > > > > > > > This issue was raised by Gabe on Nov 27 email and today also, so I > > hope > > > > we > > > > > take that into account. > > > > > It feels like gem5 is getting towards privatization... > > > > > > > > > > It’s just a opinion, it may be wrong. > > > > > > > > > > On Wed, Dec 11, 2019 at 10:12 PM Gabe Black <gabebl...@google.com> > > > > wrote: > > > > > > > > > > > Could you please clarify the policy on breaking APIs? It makes > > sense > > > > for > > > > > > releases to maintain stable APIs, but how does that apply to the > > > > > > development branch? I'm worried that it will be very hard to make > > > > changes > > > > > > that don't change any interfaces, and we definitely don't want to > > > > encourage > > > > > > a style of development where we just add and add and add without > > ever > > > > > > refactoring or consolidating things. If APIs can continue to > change > > > as > > > > > > needed in the development branch and we just need to warn people > > > before > > > > > > they're released, that seems reasonable. > > > > > > > > > > > > In general (and not necessarily tied to this proposal), we should > > try > > > > very > > > > > > hard not to have silos where people develop incompatible features > > and > > > > don't > > > > > > add to gem5 in a holistic way. It's easiest to wall off > > functionality > > > > > > behind ifdefs or config options or in separate repositories and > > then > > > > just > > > > > > ignore it when implementing new features, but that creates a lot > of > > > > > > technical debt and complexity which really cripples both gem5 and > > > > future > > > > > > development and compounds exponentially over time. > > > > > > > > > > > > Back to the branches, I'm still not a fan. I think it gives > people > > an > > > > > > excuse not to make changes ("you want me to refactor 18 months of > > > code > > > > for > > > > > > that?" (not an actual quote)), on top of other issues. Also there > > are > > > > more > > > > > > than two options. You can do your work on top of tree like I do, > > even > > > > when > > > > > > adding big new features. It ensures that you don't get too > invested > > > in > > > > bad > > > > > > paths and don't have a mountain of integration work to do when > you > > > > finally > > > > > > decide to bridge back in. It also gives the community an > > appropriate > > > > > > opportunity to provide feedback through review which is not > > practical > > > > when > > > > > > dealing with dozens of files and thousands of lines of code with > > > months > > > > of > > > > > > yet unseen history and iteration. > > > > > > > > > > > > Gabe > > > > > > > > > > > > On Wed, Dec 11, 2019 at 5:30 PM Jason Lowe-Power < > > > ja...@lowepower.com> > > > > > > wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > Here are the specific changes we're proposing. Please give us > > your > > > > > > > feedback! If you disagree with some part of this proposal, let > us > > > > know > > > > > > > why and what you suggest instead. > > > > > > > > > > > > > > # What's changing now > > > > > > > > > > > > > > 1) We are going to release gem5 19 "soon" (two blocking issues: > > 1) > > > > > > > this proposal and 2) the website) > > > > > > > 2) When we release gem5 19 we are going to create a new branch > > > called > > > > > > > "develop" which branches at the point we make the release. > > > > > > > 3) We are going to tag master with "19.0.0-beta" > > > > > > > > > > > > > > After this, all development will move to the develop branch > > instead > > > > of > > > > > > > master. This will allow the common case of people using gem5 > for > > > > > > > research to be the default. When cloning gem5 you will get a > > > "stable" > > > > > > > release. > > > > > > > > > > > > > > ## Changes for developers > > > > > > > > > > > > > > There will be two changes for developers: > > > > > > > 1) You'll have to checkout develop after cloning > > > > > > > 2) You'll have to push to "refs/for/develop" instead of > > > > "refs/for/master" > > > > > > > > > > > > > > # Next release > > > > > > > > > > > > > > When we get to our next release (gem5 20, yay!), we will merge > > the > > > > > > > develop branch into master. Assuming no hotfixes, this will be > a > > > > > > > simple fast forward. > > > > > > > > > > > > > > We will have a major release once per year usually in April > > (after > > > > the > > > > > > > MICRO deadline but with enough lead time to put together an > ISCA > > > > > > > tutorial). We will have two minor releases later in the year, > one > > > in > > > > > > > August and one in December. > > > > > > > > > > > > > > # Why are we going to releases? > > > > > > > > > > > > > > It's important for our *users* (remember, we're developing gem5 > > to > > > be > > > > > > > used by other people, not just core developers!) to be able to > > > build > > > > > > > off of stable APIs. Therefore, between major releases *we will > > not > > > > > > > change any APIs*. Code that we expect other people will build > off > > > of > > > > > > > are APIs. This includes, but is not limited to: > > > > > > > - Ports > > > > > > > - ExecContext > > > > > > > - Packet > > > > > > > - SimObject, ClockedObject, etc. > > > > > > > - Event, EventQueue, etc. > > > > > > > - Command line options > > > > > > > - And probably many more. We will work on finalizing this over > > the > > > > > > > next couple of months before the gem5-20 release. > > > > > > > > > > > > > > ## What does going to releases mean for developers? > > > > > > > > > > > > > > Thus, if any developers want to make changes to the above code, > > we > > > > must > > > > > > > Mark the API as "deprecated" at least by the *.2 December > release > > > to > > > > > > > give users at least one quarter to prepare. > > > > > > > AND either > > > > > > > 1) Make backwards compatible changes (e.g., just add a new > > function > > > > to > > > > > > > the API that we will switch to when the API merge window opens > > > (after > > > > > > > the December minor release)) > > > > > > > or 2) Create a new branch from develop that we wait to merge > > until > > > > the > > > > > > > merge window. > > > > > > > > > > > > > > # Feature branches > > > > > > > > > > > > > > Despite the disagreement in this thread so far, we're still > > > proposing > > > > > > > feature branches. However, they should be *rare*, and only > > > > maintainers > > > > > > > will have the ability to add new branches. The exact gerrit > > > > mechanisms > > > > > > > for this need to be figured out. > > > > > > > > > > > > > > We *require* something like feature branches given this new > > release > > > > > > > model for two reasons: > > > > > > > 1) There will be API breaking changes that will be long-lived > (at > > > > > > > least between a major release and the next API change merge > > > window). > > > > > > > 2) Many people develop new features to incorporate into gem5 > and > > > > > > > frequently they take months of *iteration* before they are > ready > > to > > > > be > > > > > > > merged. AMD's GPU model in the gcn-staging branch is one > example > > > > > > > ( > > > > > > > > > > > > > > > > > > > > > > > > > > > https://gem5.googlesource.com/amd/gem5/+/refs/heads/agutierr/master-gcn3-staging > > > > > > > ), > > > > > > > the FlexCPU model my group has been working on is another > > > > > > > (https://github.com/darchr/gem5/tree/flexcpu). > > > > > > > We need an improved method of merging these kinds of changes. > > > > > > > > > > > > > > The way I see it, we have two options: continue development > > locally > > > > in > > > > > > > our own repos as we are doing right now. OR we could move the > > > > > > > development *into the open* by creating feature branches on the > > > main > > > > > > > gem5 repo. I prefer the latter. There's little difference to > > > Gerrit. > > > > > > > The main difference is what you get when you do git pull. > > > > > > > > > > > > > > It will be up to the maintainers to make sure branches are > merged > > > in > > > > a > > > > > > > timely fashion and we don't create too many branches. They > should > > > be > > > > > > > kept up to date by their maintainer by frequently merging the > > > develop > > > > > > > branch. > > > > > > > > > > > > > > # Hotfixes to the stable/master branch > > > > > > > > > > > > > > It's likely we'll find bugs while developing gem5. If the > > > maintainer > > > > > > > for the component thinks the bug is severe enough (I don't want > > to > > > > > > > make any specific guidelines right now), we will backport that > > bug > > > to > > > > > > > the master branch. We'll have to work on our process for this > to > > > > > > > ensure that develop will cleanly merge into master later, but > > this > > > > > > > should be straightforward. > > > > > > > > > > > > > > # Feedback > > > > > > > > > > > > > > Please let us know what you think! We're going to go one more > > round > > > > of > > > > > > > feedback on the gem5-dev list, then we'll post the finalized > > > version > > > > > > > on gem5-users for a last round of feedback. I'm hoping to begin > > to > > > > > > > implement this early next year! > > > > > > > > > > > > > > Thanks for your patience on this. I know it's a lot to read > > through > > > > > > > and think about, but it's important that we get this process > > right > > > > the > > > > > > > first time. We don't want to have to change this every year :). > > > > > > > > > > > > > > Cheers, > > > > > > > Jason > > > > > > > > > > > > > > On Tue, Dec 10, 2019 at 4:49 PM Bobby Bruce < > bbr...@ucdavis.edu> > > > > wrote: > > > > > > > > > > > > > > > > Thank you all for your comments on this. > > > > > > > > > > > > > > > > In regards to the master/develop branch idea, I'm a big > > advocate > > > of > > > > > > > Jason's > > > > > > > > "make the most common use the easiest" philosophy. I think he > > > did a > > > > > > good > > > > > > > > job of describing why this is important. We use the git repo > as > > > our > > > > > > > > primarily way of distributing gem5, and, as such, I believe > the > > > > default > > > > > > > > behaviour should be that when a user clones the repo, they > are > > > > getting > > > > > > > the > > > > > > > > latest stable release to build on top of. I strongly suspect > > > users > > > > will > > > > > > > > disproportionately use whatever `git clone` gives them rather > > > than > > > > move > > > > > > > > between branches or tags in order to get specific, stable > > > releases. > > > > If > > > > > > > most > > > > > > > > users end up working on the very latest "unstable" > development > > > > version > > > > > > > then > > > > > > > > our release system becomes pointless. I realize as everyone > on > > > this > > > > > > list > > > > > > > is > > > > > > > > a gem5 contributor, there is a preference for the development > > > > branch to > > > > > > > be > > > > > > > > default, but I'd hope we are in the minority of people who > > > actually > > > > use > > > > > > > > gem5. My vote therefore still goes to having two branches: > the > > > > > > > > master/"release" branch and the "develop" branch for > day-to-day > > > > > > > > development. Hotfixes would be created on a separate branch > and > > > > merged > > > > > > > into > > > > > > > > both upon completion. > > > > > > > > > > > > > > > > Regarding feature branching --- My intention with this part > of > > > the > > > > > > > proposal > > > > > > > > was not for feature branches to be the default manner in > which > > > > people > > > > > > > > commit to gem5 (as they would be if we were to implement the > > > > GitFlow > > > > > > > model > > > > > > > > in full), I consider them something we should allow, in some > > > form, > > > > to > > > > > > > > improve the development process in certain circumstances, but > > > that > > > > they > > > > > > > > should be a rather rare thing. For example, a new component > may > > > > need > > > > > > > > several months of development work, consisting of many > commits. > > > > > > > Developers, > > > > > > > > at present, have two options: either A) gradually incorporate > > > > commits > > > > > > > over > > > > > > > > the months on the master branch, or B) push all the commits > to > > > the > > > > > > master > > > > > > > > branch in one go. I don't think either are ideal solutions. > A > > > > > > dedicated > > > > > > > > feature branch would quarantine these changes while in active > > > > > > > development, > > > > > > > > to be merged at a later date. A second circumstance that > could > > > > arise is > > > > > > > > when we wish to postpone the incorporation of a feature. A > > change > > > > that > > > > > > > > requires an API alteration, for example, should be held-off > > until > > > > we > > > > > > are > > > > > > > > ready to deploy a major release. Isolating such a change in a > > > > feature > > > > > > > > branch, to be merged later, may be necessary. This all being > > > said, > > > > I > > > > > > > > acknowledge there is a risk in code getting lost > indefinitely, > > or > > > > for > > > > > > > long > > > > > > > > periods of time, in feature branches, and our repo getting > > messy > > > as > > > > a > > > > > > > > result. After some discussion with Jason, I believe the best > of > > > > both > > > > > > > worlds > > > > > > > > can be achieved if we make feature branches "by request" --- > > > anyone > > > > > > > wanting > > > > > > > > to create a feature branch would need to ask a maintainer > (or a > > > > PCM) to > > > > > > > > create one for them. Whether or not a feature branch is to be > > > > created > > > > > > > would > > > > > > > > be decided upon on a case-by-case basis. > > > > > > > > > > > > > > > > I hope no one feels like I'm dictating anything here. I'm > > writing > > > > this > > > > > > > > primarily to prompt more discussion, so please feel free to > > > counter > > > > any > > > > > > > of > > > > > > > > my points or suggest alternatives. Simple messages of > support, > > as > > > > well > > > > > > as > > > > > > > > criticism, would also be appreciated to help us get a feel > for > > > what > > > > is, > > > > > > > or > > > > > > > > what is not, proving popular with the community. > > > > > > > > > > > > > > > > Kind regards, > > > > > > > > Bobby > > > > > > > > -- > > > > > > > > Dr. Bobby R. Bruce > > > > > > > > Room 2235, > > > > > > > > Kemper Hall, UC Davis > > > > > > > > Davis, > > > > > > > > CA, 95616 > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 27, 2019 at 4:49 PM Gabe Black < > > gabebl...@google.com > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > The idea of feature branches makes me very nervous. It > sounds > > > > like a > > > > > > > dark > > > > > > > > > corner for a large body of code to grow in, and then once > it > > > pops > > > > > > back > > > > > > > into > > > > > > > > > the light there may be very ingrained and difficult to fix > > > > problems > > > > > > > which > > > > > > > > > either have to be ignored or fixed. A long standing branch > > may > > > > also > > > > > > be > > > > > > > very > > > > > > > > > hard to reconcile with ToT, and it may be very tempting to > > just > > > > let > > > > > > it > > > > > > > die > > > > > > > > > rather than try to jump that energy barrier and get it back > > > into > > > > > > > mainline. > > > > > > > > > > > > > > > > > > This would basically be sanctifying the practice of > > > perma-forking > > > > > > gem5, > > > > > > > > > which unfortunately happens anyway now and then. There have > > > been > > > > a > > > > > > > number > > > > > > > > > of times on the mailing list someone says something along > the > > > > lines > > > > > > of > > > > > > > "I > > > > > > > > > downloaded special version of gem5 foo from some github" > and > > it > > > > turns > > > > > > > out > > > > > > > > > it's X years old with a bunch of old and new bugs, nobody > > knows > > > > how > > > > > > it > > > > > > > > > works, etc. Let's not encourage that :-). > > > > > > > > > > > > > > > > > > Gabe > > > > > > > > > > > > > > > > > > On Wed, Nov 27, 2019 at 9:17 AM Vince Harron < > > > vhar...@google.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > I agree with Ciro that the de facto standard used by most > > > > projects > > > > > > is > > > > > > > > > that > > > > > > > > > > mainline development should happen in master and releases > > > > should be > > > > > > > new > > > > > > > > > > branches created off of master. > > > > > > > > > > > > > > > > > > > > You could also create a branch “stable” or “latest” that > > gets > > > > > > > updated to > > > > > > > > > > the latest release. > > > > > > > > > > > > > > > > > > > > The default download instructions could say something > like > > > > > > > > > > > > > > > > > > > > git clone <url> —branch stable > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 27, 2019 at 8:54 AM Jason Lowe-Power < > > > > > > > ja...@lowepower.com> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Thanks for the input Ciro! > > > > > > > > > > > > > > > > > > > > > > My pushback here is that most people who use gem5 will > be > > > > making > > > > > > > > > > > modifications to it (at least for the foreseeable > > future). > > > I > > > > > > don't > > > > > > > > > think > > > > > > > > > > we > > > > > > > > > > > want to common use case to be 1) Download a tarball, 2) > > > make > > > > > > local > > > > > > > > > > > modifications to the source. I think we want people to > 1) > > > > clone a > > > > > > > > > > "stable" > > > > > > > > > > > gem5, 2) make modifications to the stable branch for > > their > > > > > > > research *in > > > > > > > > > > > git*. > > > > > > > > > > > > > > > > > > > > > > My experience with architecture grad students and > > > researchers > > > > is > > > > > > > that > > > > > > > > > if > > > > > > > > > > > you don't force them to use things like git most of > them > > > > won't. > > > > > > > One of > > > > > > > > > my > > > > > > > > > > > (personal) goals is to make it easy/default to use gem5 > > > "the > > > > > > right > > > > > > > > > way". > > > > > > > > > > > > > > > > > > > > > > In fact, I almost don't want to provide tarballs at > all. > > > > Unless > > > > > > we > > > > > > > can > > > > > > > > > > > provide pre-build gem5 binaries, I don't think it makes > > > sense > > > > to > > > > > > > > > provide > > > > > > > > > > > tarballs. I don't think prebuilt binaries make sense > for > > > the > > > > > > > research > > > > > > > > > use > > > > > > > > > > > case of gem5 since almost all research will require > > > extending > > > > > > gem5 > > > > > > > and > > > > > > > > > > > there's no way to dynamically load in new modules > (though > > > we > > > > > > should > > > > > > > > > > support > > > > > > > > > > > this in the future!). For teaching with gem5 prebuilt > > > > binaries > > > > > > > could > > > > > > > > > make > > > > > > > > > > > sense, but I think we'll hold off on that for now, TBH. > > > > > > > > > > > > > > > > > > > > > > I think this comes down to "make the common case > fast". I > > > > believe > > > > > > > the > > > > > > > > > > > common case is researchers who want to make minor > changes > > > to > > > > gem5 > > > > > > > > > (e.g., > > > > > > > > > > > add a new SimObject or modify the cache protocol) and > run > > > > > > > experiments > > > > > > > > > for > > > > > > > > > > > their research. Developing gem5 for the community > (using > > > the > > > > > > > > > development > > > > > > > > > > > branch) is actually the uncommon case. > > > > > > > > > > > > > > > > > > > > > > BTW, I agree with you that we should try to follow > > whatever > > > > > > > practices > > > > > > > > > are > > > > > > > > > > > most popular when it makes sense. However, in many ways > > > gem5 > > > > is a > > > > > > > > > unique > > > > > > > > > > > project, so following others doesn't always make sense > > :). > > > > > > > > > > > > > > > > > > > > > > I'm open to discussion on this. Thoughts? > > > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > Jason > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 27, 2019 at 12:01 AM Ciro Santilli < > > > > > > > ciro.santi...@arm.com> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Thanks Bobby for pushing this forward, having > releases > > > > would > > > > > > be a > > > > > > > > > good > > > > > > > > > > > > thing for gem5. > > > > > > > > > > > > > > > > > > > > > > > > I would recommend against having both master and > > develop > > > > branch > > > > > > > > > though, > > > > > > > > > > > > because in the large majority of projects out there, > > > master > > > > == > > > > > > > the > > > > > > > > > > latest > > > > > > > > > > > > version, so I'm always confused when I have to switch > > to > > > > > > develop, > > > > > > > > > > > specially > > > > > > > > > > > > if it isn't the default HEAD (is is however possible > to > > > set > > > > the > > > > > > > > > default > > > > > > > > > > > > branch to develop as shown as: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://gerrit-review.googlesource.com/Documentation/project-configuration.html#default-branch > > > > > > > > > > > > ) > > > > > > > > > > > > > > > > > > > > > > > > This is even more important if we start having > prebuilt > > > > > > releases > > > > > > > or > > > > > > > > > > tgzs > > > > > > > > > > > > with source (which are smaller than full repo) for > the > > > > tags, at > > > > > > > which > > > > > > > > > > > point > > > > > > > > > > > > basically everyone who clones wants the latest. > > > > > > > > > > > > > > > > > > > > > > > > I would instead recommend: > > > > > > > > > > > > > > > > > > > > > > > > - master: latest version > > > > > > > > > > > > - v2019: a tag. The latest one can be easily found > > with: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://stackoverflow.com/questions/1404796/how-to-get-the-latest-tag-name-in-current-branch-in-git > > > > > > > > > > > > - 2019 or b2019: branch created at the same time as > > v2019 > > > > to > > > > > > > which > > > > > > > > > > > > backports are applied, and to which v2019.0.1 tags > can > > > also > > > > be > > > > > > > > > > optionally > > > > > > > > > > > > applied > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ________________________________ > > > > > > > > > > > > From: gem5-dev <gem5-dev-boun...@gem5.org> on behalf > > of > > > > Bobby > > > > > > > Bruce > > > > > > > > > < > > > > > > > > > > > > bbr...@ucdavis.edu> > > > > > > > > > > > > Sent: Wednesday, November 27, 2019 12:10 AM > > > > > > > > > > > > To: gem5-dev@gem5.org <gem5-dev@gem5.org> > > > > > > > > > > > > Subject: [gem5-dev] gem5 19.0.0 : New git branching > > > > proposal > > > > > > > > > > > > > > > > > > > > > > > > Dear all, > > > > > > > > > > > > > > > > > > > > > > > > As you know, at the end of this quarter we will be > > > > releasing > > > > > > > gem5-19, > > > > > > > > > > the > > > > > > > > > > > > first official release of gem5. As part of this > release > > > > we'd > > > > > > > like to > > > > > > > > > > > change > > > > > > > > > > > > our git branching structure. Therefore, I'm writing > to > > > ask > > > > for > > > > > > > > > feedback > > > > > > > > > > > on > > > > > > > > > > > > what we have planned and whether it can be improved > > upon. > > > > > > > > > > > > > > > > > > > > > > > > We'd like to have a git repo structure similar to > that > > > used > > > > in > > > > > > > > > gitflow > > > > > > > > > > > > development: > > > > > > > https://nvie.com/posts/a-successful-git-branching-model > > > > > > > > > . > > > > > > > > > > > > I've > > > > > > > > > > > > seen this model work well before, as it has some > > > worthwhile > > > > > > > > > abstractors > > > > > > > > > > > for > > > > > > > > > > > > public, open source git projects with regular > releases. > > > > What > > > > > > I'd > > > > > > > like > > > > > > > > > > to > > > > > > > > > > > > incorporate from this model is the following: > > > > > > > > > > > > > > > > > > > > > > > > - Two permanent branches: master and develop. > > > > > > > > > > > > - "develop" would function as master does now. This > > would > > > > be > > > > > > the > > > > > > > main > > > > > > > > > > > point > > > > > > > > > > > > in which changes are applied between gem5 releases. > > > > > > > > > > > > - Upon a new release of gem5, the develop branch > would > > be > > > > > > merged > > > > > > > into > > > > > > > > > > > > master and a new git tag added to master indicating > the > > > > release > > > > > > > > > > version. > > > > > > > > > > > > Ergo, the master branch would always contain the > latest > > > > release > > > > > > > of > > > > > > > > > > gem5. > > > > > > > > > > > > - If a quick hotfix is needed, a new "hotfix" branch > > > would > > > > be > > > > > > > created > > > > > > > > > > and > > > > > > > > > > > > merged into both the develop and master branches upon > > > > > > completion. > > > > > > > > > This > > > > > > > > > > > > would also require a new tag on the master branch. (I > > > > suggest > > > > > > > using > > > > > > > > > the > > > > > > > > > > > > standard "Version [Major].[Minor].[Hotfix]" version > > > > numbering > > > > > > > system. > > > > > > > > > > > I.e., > > > > > > > > > > > > the first version would be V19.0.0, a hotfix to this > > > would > > > > make > > > > > > > it > > > > > > > > > > > V19.0.1, > > > > > > > > > > > > and a minor release would make it V19.1.0). > > > > > > > > > > > > - The creation of feature branches would be > permitted. > > > > These > > > > > > > branches > > > > > > > > > > > would > > > > > > > > > > > > encapsulate the gradual development of large features > > > > (i.e., > > > > > > ones > > > > > > > > > > carried > > > > > > > > > > > > out over many commits). When complete a feature > branch > > > > would be > > > > > > > > > merged > > > > > > > > > > > into > > > > > > > > > > > > the develop branch. They'd be no obligation to use > > > feature > > > > > > > branches > > > > > > > > > > > though > > > > > > > > > > > > we believe they could be of value in certain cases. > For > > > > > > example, > > > > > > > if a > > > > > > > > > > > > developer wishes to postpone a developed feature for > a > > > > given > > > > > > gem5 > > > > > > > > > > release > > > > > > > > > > > > (e.g., something more suited for a major release > rather > > > > than a > > > > > > > minor > > > > > > > > > > > one), > > > > > > > > > > > > then they could submit their changes as a feature > > branch > > > > and > > > > > > > wait to > > > > > > > > > > > merge > > > > > > > > > > > > to the develop branch at a later date. > > > > > > > > > > > > > > > > > > > > > > > > I believe this setup would make our development > process > > > run > > > > > > > smoother > > > > > > > > > > and > > > > > > > > > > > > give gem5 users more stability. Day-to-day > development > > > > wouldn't > > > > > > > > > change > > > > > > > > > > > much > > > > > > > > > > > > as committing to the develop branch would work in the > > > same > > > > way > > > > > > as > > > > > > > > > > > > submitting to master does now. > > > > > > > > > > > > > > > > > > > > > > > > If anyone has any thoughts about this, I'd be happy > to > > > hear > > > > > > from > > > > > > > you. > > > > > > > > > > > > > > > > > > > > > > > > Kind regards, > > > > > > > > > > > > Bobby > > > > > > > > > > > > -- > > > > > > > > > > > > Dr. Bobby R. Bruce > > > > > > > > > > > > Room 2235, > > > > > > > > > > > > Kemper Hall, UC Davis > > > > > > > > > > > > Davis, > > > > > > > > > > > > CA, 95616 > > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > > gem5-dev mailing list > > > > > > > > > > > > gem5-dev@gem5.org > > > > > > > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev > > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > > gem5-dev mailing list > > > > > > > > > > > > gem5-dev@gem5.org > > > > > > > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > gem5-dev mailing list > > > > > > > > > > > gem5-dev@gem5.org > > > > > > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > Vince Harron | Engineering Manager | > vhar...@google.com > > | > > > > > > > > > 858-442-0868 <(858)%20442-0868> > > > > > > > > > > <(858)%20442-0868> > > > > > > > > > > _______________________________________________ > > > > > > > > > > gem5-dev mailing list > > > > > > > > > > gem5-dev@gem5.org > > > > > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev > > > > > > > > > _______________________________________________ > > > > > > > > > gem5-dev mailing list > > > > > > > > > gem5-dev@gem5.org > > > > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev > > > > > > > > _______________________________________________ > > > > > > > > gem5-dev mailing list > > > > > > > > gem5-dev@gem5.org > > > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev > > > > > > > _______________________________________________ > > > > > > > gem5-dev mailing list > > > > > > > gem5-dev@gem5.org > > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev > > > > > > _______________________________________________ > > > > > > gem5-dev mailing list > > > > > > gem5-dev@gem5.org > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev > > > > > _______________________________________________ > > > > > gem5-dev mailing list > > > > > gem5-dev@gem5.org > > > > > http://m5sim.org/mailman/listinfo/gem5-dev > > > > _______________________________________________ > > > > gem5-dev mailing list > > > > gem5-dev@gem5.org > > > > http://m5sim.org/mailman/listinfo/gem5-dev > > > _______________________________________________ > > > gem5-dev mailing list > > > gem5-dev@gem5.org > > > http://m5sim.org/mailman/listinfo/gem5-dev > > _______________________________________________ > > gem5-dev mailing list > > gem5-dev@gem5.org > > http://m5sim.org/mailman/listinfo/gem5-dev > _______________________________________________ > gem5-dev mailing list > gem5-dev@gem5.org > http://m5sim.org/mailman/listinfo/gem5-dev _______________________________________________ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
