Hi devs,
A lot has been said and it is difficult to address most of the interesting
opinions that have been raised throughout the conversation.
In general I am a moderate fan of releases and I strongly agree with Daniel's
story:
There is a lot of discussion about when users should be updating their gem5
versions, but how many even want to update? "You don't change a winning team"
is something that I've heard a lot from teachers and colleagues during my
former studies: why would you risk updating and breaking something, or reducing
performance improvements? Throughout the years I've seen that students (me
included) tend not to update anything (OS, compiler, software being used) until
their work is done out of fear of having to reconfigure/fix things.
I got asked several times which stable version should someone use. It is very
weird and scary for a user who's willing to rebase his work to get a commit Id
as an answer.
Especially if there is no bug-fixing happening on a commitId. In fact, if you
don't backport bug fixes, the entire concept of stableness is flawed, since you
might discover a new
bug later on and you are basically forcing a USER (not an active developer) to
rebase his work in order to integrate the bug.
(A user could cherry-pick the commit on his own, but first he needs to be aware
that a specific commit is fixing his problem)
I think we all agree that having a linear chunck of untagged and unbranched
commits makes it difficult for common users to understand what they are getting
when they rebase their work. It is not for some of us maybe, spending most of
our engineering time reviewing patches and contributing to gerrit. I could more
or less tell what's the difference in the simulator
between today's gem5 and last month gem5, but that doesn't apply to the rest of
the people.
Giacomo
________________________________
From: gem5-dev <gem5-dev-boun...@gem5.org> on behalf of Daniel Carvalho
<oda...@yahoo.com.br>
Sent: 13 December 2019 11:32
To: gem5 Developer List <gem5-dev@gem5.org>
Subject: Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
A bit late to the party, but here are my two cents.
Regarding developing, based on previous experiences and questions I've received
from users throughout these years, I couldn't agree more with Jason's sentence:
"My experience with architecture grad students and researchers is that if you
don't force them to use things like git most of them won't".
I've also asked some of these users their preference, and the stable version on
master with a development branch was an unanimous pick. However, it seems that
this choice was made mostly because it'd require less learning effort; and I am
positive not needing to know anything about git was appealing to some (i.e.,
"git clone" would be the first and last git command they'd use). Therefore, I
am not sure this approach will achieve the intended goal of "directing to
correct usage".
There is a lot of discussion about when users should be updating their gem5
versions, but how many even want to update? "You don't change a winning team"
is something that I've heard a lot from teachers and colleagues during my
former studies: why would you risk updating and breaking something, or reducing
performance improvements? Throughout the years I've seen that students (me
included) tend not to update anything (OS, compiler, software being used) until
their work is done out of fear of having to reconfigure/fix things.
In the end I think that the focus of stable versions should be on
reproducibility, not on updatability: researchers will inform which gem5 tag
they are using, making it less difficult for readers to reproduce their
results. There will obviously be users updating, but changing from possibly
having to rebase on every commit to once or twice a year seems like a
reasonable compromise given the benefits; the versioning will make updates
start to feel more like upgrades, and API changes switch from a constant fear
to a short term agreement. There is no need to say that in any case API changes
must be made carefully and as infrequently as possible.
Personally, I think major and minor differences should not differ as of now;
this would complicate reviewing and merging, adding a "when should this go?" to
an already complex process. I am not saying this is not possible, but maybe
it'd be aiming too high and it would be better to take smaller steps.
Overall, the ideas are excellent, and if they manage to leave the theory plane,
it will be a great improvement! Looking forward to what's in the future.
Regards,
Daniel
Em sexta-feira, 13 de dezembro de 2019 02:16:05 GMT+1, Jason Lowe-Power
<ja...@lowepower.com> escreveu:
Ok... so are you suggesting there be no difference between major and minor
releases?
How do you suggest communicating that API changes are coming? The most
common way to do this is by marking functions as deprecated for some period
of time.
Thanks,
Jason
On Thu, Dec 12, 2019 at 4:26 PM Gabe Black <gabebl...@google.com> wrote:
> I think that's what the release vs. development branch split already
> provides. If you want stable interfaces, then the release branch will stay
> as it is for a year, or forever if you don't move to new releases.
>
> Gabe
>
> On Thu, Dec 12, 2019 at 4:18 PM Jason Lowe-Power <ja...@lowepower.com>
> wrote:
>
> > Thanks Gabe!
> >
> > I think we're mostly in agreement. The key disagreement is the following:
> >
> > Note that I'm not advocating for a wild west free for all with no
> > > accountability, but only being able to make changes to "interfaces"
> > (which
> > > pretty much everything could be considered) once a year is much too far
> > in
> > > the other direction, especially on a so called development branch.
> >
> >
> > I believe, and I've heard from many gem5 users, that we need to have more
> > stable interfaces. I think that once a year is a good compromise, but I'm
> > open to a faster cadence. Either way, we must give a heads up of at least
> > months to change an interface.
> >
> > Can you propose a different idea that will satisfy our need to both
> > provided stability and allow developers to update code?
> >
> > Alternatively, maybe we need to come up with a better transition than
> just
> > "tomorrow you can't modify interfaces." Do you have any ideas on a more
> > smooth transition to this development model of providing stability to our
> > users?
> >
> > Thanks,
> > Jason
> >
> > On Thu, Dec 12, 2019 at 4:03 PM Gabe Black <gabebl...@google.com> wrote:
> >
> > > I'll point out that nobody had to rebase O3 when, for example, the
> > > ExecContext changed, because it was already in the tree and was changed
> > > along with everything else.
> > >
> > > I think you have an important contradiction in your reasoning above,
> > namely
> > > that gem5 is simultaneously too unstable and too stable. You're saying
> > it's
> > > too unstable since the interfaces change too often, but then also say
> > that
> > > there have been very few new features in gem5 in the last few years.
> > >
> > > Fundamentally, new features require changing things. If we clamp down
> on
> > > changing what exists or add a bunch of onerous requirements, then there
> > > will either be no change, or it will be side stepped by laying down a
> > layer
> > > of concrete and adding something new over the top, ie ifdefs, config
> > > options, "real" versions of APIs which live alongside the old ones.
> These
> > > are all things which have already happened in gem5 and which I've spent
> > > some time partially cleaning up. In the real world, see the x86 ISA as
> an
> > > example. Do I really need to be able to boot my huge workstation into
> > > little bitty 16 bit real mode? Probably not, but I still could, and the
> > ISA
> > > is a bit of a mess because of it.
> > >
> > > Frankly, there is also a lot of technical debt in gem5. A lot of things
> > > were accreted over time, often by well meaning but not necessary expert
> > > programmers, including earlier (and even current) versions of me, and
> > those
> > > things need to be cleaned up. Fixing those things requires changing
> > > bad/obsolete/etc interfaces, restructuring code, and generally moving
> > > things around which have ripple effects throughout the simulator. You
> > can't
> > > make lemonade without cracking a few lemons.
> > >
> > > I think a few things are key:
> > > Tests make changes less dangerous. We need more of these.
> > > Warn people about changes so they have an opportunity to update and
> don't
> > > get blindsided.
> > > Change interfaces carefully, with consideration for the future so they
> > > don't have to be changed excessively.
> > >
> > > Note that I'm not advocating for a wild west free for all with no
> > > accountability, but only being able to make changes to "interfaces"
> > (which
> > > pretty much everything could be considered) once a year is much too far
> > in
> > > the other direction, especially on a so called development branch.
> > >
> > > Also, I think everyone here is well intentioned and wants to make gem5
> > > available to as many people as possible and as useful to them as
> > possible,
> > > and we're just trying to figure out how to do that. That's why we're
> > > hashing this out on a public mailing list and not in a private thread,
> > > secret meetings, etc.
> > >
> > > Gabe
> > >
> > > On Thu, Dec 12, 2019 at 10:08 AM Jason Lowe-Power <ja...@lowepower.com
> >
> > > wrote:
> > >
> > > > Hey Abhishek,
> > > >
> > > > Yes! Not only will we be releasing disk images (that we can depending
> > on
> > > > license for the benchmarks) and kernel images, we will also be
> > releasing
> > > > all of the documentation and the scripts describing how these images
> > were
> > > > created. These will also be updated at every release as appropriate.
> > > >
> > > > Cheers,
> > > > Jason
> > > >
> > > >
> > > >
> > > > On Thu, Dec 12, 2019 at 10:04 AM Abhishek Singh <
> > > > abhishek.singh199...@gmail.com> wrote:
> > > >
> > > > > Hello Jason,
> > > > >
> > > > > This is perfect !
> > > > >
> > > > > I had one more question, for full system simulations will there be
> > > > release
> > > > > of images and kernel files for every architectures (arm , x86,
> etc)?
> > > > >
> > > > > On Thu, Dec 12, 2019 at 12:57 PM Jason Lowe-Power <
> > ja...@lowepower.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hey Abhishek,
> > > > > >
> > > > > > Emails will continue to gem5-dev on every changeset pushed to the
> > > > develop
> > > > > > branch as they do now to master :). We can discuss if we want the
> > > same
> > > > > for
> > > > > > feature branches (if we end up using them).
> > > > > >
> > > > > > Your first interpretation on gem5 stable is correct (sorry if
> this
> > > > wasn't
> > > > > > clear). It will be much more heavily tested than the
> > minute-by-minute
> > > > > > releases from the develop branch. With this testing we will be
> > > > publishing
> > > > > > the following:
> > > > > > - What (common) workloads are supported (e.g., SPEC, parsec,
> etc.).
> > > > Which
> > > > > > workloads we use here will be discussed in the future, stay
> tuned.
> > > > > > - For all of the workloads, we will publish common statistics
> for a
> > > few
> > > > > > different systems (e.g., time to simulate, IPC, cache miss rates,
> > > > memory
> > > > > > bandwidth, etc). The systems used and the stats will be discussed
> > in
> > > > the
> > > > > > future.
> > > > > >
> > > > > > The stable release will *not* be a continuous process. The only
> > time
> > > > the
> > > > > > stable branch will be updated is 1) At releases or 2) if a
> "major"
> > > bug
> > > > is
> > > > > > encountered. For non-release updates (e.g., for bugs), we'll be
> > very
> > > > > > careful to either re-validate our results or somehow know the
> > results
> > > > > won't
> > > > > > change.
> > > > > >
> > > > > > Cheers,
> > > > > > Jason
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Dec 12, 2019 at 9:38 AM Abhishek Singh <
> > > > > > abhishek.singh199...@gmail.com> wrote:
> > > > > >
> > > > > > > Hi Jason,
> > > > > > >
> > > > > > > Thanks for your email. It cleared a lot of misunderstanding
> > which I
> > > > > had.
> > > > > > >
> > > > > > > Is it possible to have an email sent on every commit we make to
> > > > atleast
> > > > > > > gem5-dev?
> > > > > > > The email list could be different and can be sent to people who
> > are
> > > > > > > interested in this so that it does not spam to gem5-Dev list.
> > > > > > >
> > > > > > > I am talking about gem5 developer branch and not stable.
> > > > > > >
> > > > > > > This is because, it will keep all the interested community
> > members
> > > > well
> > > > > > > informed about new features that are added, who like to keep
> note
> > > of
> > > > > > latest
> > > > > > > changes and merge in their projects as required.
> > > > > > >
> > > > > > > And the way, I understand that a stable gem5 when a user
> desires
> > > is:
> > > > > > > It wants the conference accepted applications (standard
> workload
> > > for
> > > > > > > example spec, parsec, etc to run without any errors on both SE
> > and
> > > FS
> > > > > > mode
> > > > > > > for every architecture and cpu models.
> > > > > > >
> > > > > > > If that’s what the stable releases are going to be testing
> before
> > > > > > releasing
> > > > > > > it, then a stable release is much more helpful and will have
> the
> > > wide
> > > > > > > reach. But when I read proposal this seems to be step by step
> > > process
> > > > > > > reaching it as the final goal. Please correct me on this if I
> > > > > understood
> > > > > > it
> > > > > > > wrong.
> > > > > > >
> > > > > > >
> > > > > > > The aim of stable releases is a continuous process and will
> > always
> > > be
> > > > > > > doubtful unless tested with all the major conference accepted
> > > > > application
> > > > > > > on every stable releases.
> > > > > > >
> > > > > > > On Thu, Dec 12, 2019 at 12:12 PM Jason Lowe-Power <
> > > > ja...@lowepower.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > First of all, let me say that I hope it's clear that gem5 is
> > not
> > > > > > > > "controlled" in any way by me! As laid out in our governance
> > > > > document (
> > > > > > > > http://gem5.org/Governance#Overview), "gem5 is a
> meritocratic,
> > > > > > > > consensus-based community project". Through these emails, and
> > as
> > > > the
> > > > > > > chair
> > > > > > > > of the project management committee, I'm trying to *build*
> > > > consensus.
> > > > > > > >
> > > > > > > > Importantly, there's a reason we're trying to make this push
> > for
> > > > > stable
> > > > > > > > APIs. I've heard from dozens of current and potential gem5
> > users
> > > > that
> > > > > > > they
> > > > > > > > want stable gem5 releases. By providing stable releases, we
> > will
> > > be
> > > > > > > > expanding the users of gem5 (and, IMO, making the research
> and
> > > > papers
> > > > > > > that
> > > > > > > > use gem5 better).
> > > > > > > >
> > > > > > > > Could you please clarify the policy on breaking APIs? It
> makes
> > > > sense
> > > > > > for
> > > > > > > > > releases to maintain stable APIs, but how does that apply
> to
> > > the
> > > > > > > > > development branch? I'm worried that it will be very hard
> to
> > > make
> > > > > > > changes
> > > > > > > > > that don't change any interfaces, and we definitely don't
> > want
> > > to
> > > > > > > > encourage
> > > > > > > > > a style of development where we just add and add and add
> > > without
> > > > > ever
> > > > > > > > > refactoring or consolidating things. If APIs can continue
> to
> > > > change
> > > > > > as
> > > > > > > > > needed in the development branch and we just need to warn
> > > people
> > > > > > before
> > > > > > > > > they're released, that seems reasonable.
> > > > > > > >
> > > > > > > >
> > > > > > > > Let's dig into this deeper. First, I'd like to remind
> everyone
> > > that
> > > > > the
> > > > > > > > current proposal was the community consensus reached in the
> > gem5
> > > > > > roadmap
> > > > > > > > document:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/1fv01HavfkIIqfcgZoKUpojkUKkWujxspRCcvS5cTfkk/edit
> > > > > > > > .
> > > > > > > > Of course, we can always change what we decided before :).
> > > > > > > >
> > > > > > > > The goal is to provide our users with a stable base to build
> of
> > > > off.
> > > > > If
> > > > > > > we
> > > > > > > > are constantly changing interfaces, like we do today, it's
> > > > impossible
> > > > > > to
> > > > > > > > build a project based off of gem5. There have been many
> > projects
> > > > that
> > > > > > > have
> > > > > > > > languished because of this including gem5+SST and dist-gem5.
> If
> > > we
> > > > > had
> > > > > > > well
> > > > > > > > defined interfaces *that weren't constantly changing* I
> believe
> > > it
> > > > > > would
> > > > > > > > make gem5 a more widely used project.
> > > > > > > >
> > > > > > > > The proposal is to have *one* API breaking release every
> year.
> > > This
> > > > > was
> > > > > > > > decided based on community feedback in the roadmap document.
> > > > > > > >
> > > > > > > > I (personally) believe we need to slow down and be more
> > > deliberate
> > > > > > about
> > > > > > > > changing our interfaces. I think that it's a good thing that
> > "it
> > > > will
> > > > > > be
> > > > > > > > very hard to make changes that don't change any interfaces".
> > This
> > > > > will
> > > > > > > make
> > > > > > > > gem5 more stable and easy for others to use. If you believe
> you
> > > > need
> > > > > to
> > > > > > > > change an interface, we should have a discussion about it
> > first,
> > > > and
> > > > > > this
> > > > > > > > should include a discussion about how the interface change
> will
> > > > > affect
> > > > > > > our
> > > > > > > > users.
> > > > > > > >
> > > > > > > > I strongly agree that "we definitely don't want to encourage
> a
> > > > style
> > > > > of
> > > > > > > > development where we just add and add and add without ever
> > > > > refactoring
> > > > > > or
> > > > > > > > consolidating things." The current proposal states that when
> > > > changing
> > > > > > an
> > > > > > > > interface, we first add a new function and mark the old
> version
> > > as
> > > > > > > > deprecated. Then, during the API changing merge window delete
> > all
> > > > of
> > > > > > the
> > > > > > > > deprecated functions. This is going to take more maintenance
> > from
> > > > our
> > > > > > > side,
> > > > > > > > but we now have the resources to do this.
> > > > > > > >
> > > > > > > > I disagree that "we just need to warn people before [API
> > changes]
> > > > are
> > > > > > > > released" is enough. We need to give them significant
> (months)
> > of
> > > > > lead
> > > > > > > > time, and we need to give them the opportunity to transition.
> > In
> > > > > fact,
> > > > > > I
> > > > > > > > believe this is what Abhishek was getting at in his follow
> up.
> > > > > > > >
> > > > > > > > In general (and not necessarily tied to this proposal), we
> > should
> > > > try
> > > > > > > very
> > > > > > > > > hard not to have silos where people develop incompatible
> > > features
> > > > > and
> > > > > > > > don't
> > > > > > > > > add to gem5 in a holistic way. It's easiest to wall off
> > > > > functionality
> > > > > > > > > behind ifdefs or config options or in separate repositories
> > and
> > > > > then
> > > > > > > just
> > > > > > > > > ignore it when implementing new features, but that creates
> a
> > > lot
> > > > of
> > > > > > > > > technical debt and complexity which really cripples both
> gem5
> > > and
> > > > > > > future
> > > > > > > > > development and compounds exponentially over time.
> > > > > > > >
> > > > > > > >
> > > > > > > > I agree with this statement. We need to encourage development
> > in
> > > > the
> > > > > > open
> > > > > > > > based on the mainline code. I strongly believe the current
> > > proposal
> > > > > > > *makes
> > > > > > > > this easier*, not harder. Today, many people do exactly what
> > > you're
> > > > > > > > describing. By implementing stable APIs that people can build
> > off
> > > > of
> > > > > > and
> > > > > > > > allowing branches, we will be making it easier to add
> features
> > to
> > > > > gem5
> > > > > > > in a
> > > > > > > > holistic way.
> > > > > > > >
> > > > > > > > Back to the branches, I'm still not a fan. I think it gives
> > > people
> > > > an
> > > > > > > > > excuse not to make changes ("you want me to refactor 18
> > months
> > > of
> > > > > > code
> > > > > > > > for
> > > > > > > > > that?" (not an actual quote)), on top of other issues. Also
> > > there
> > > > > are
> > > > > > > > more
> > > > > > > > > than two options. You can do your work on top of tree like
> I
> > > do,
> > > > > even
> > > > > > > > when
> > > > > > > > > adding big new features. It ensures that you don't get too
> > > > invested
> > > > > > in
> > > > > > > > bad
> > > > > > > > > paths and don't have a mountain of integration work to do
> > when
> > > > you
> > > > > > > > finally
> > > > > > > > > decide to bridge back in. It also gives the community an
> > > > > appropriate
> > > > > > > > > opportunity to provide feedback through review which is not
> > > > > practical
> > > > > > > > when
> > > > > > > > > dealing with dozens of files and thousands of lines of code
> > > with
> > > > > > months
> > > > > > > > of
> > > > > > > > > yet unseen history and iteration.
> > > > > > > >
> > > > > > > >
> > > > > > > > No one can do what you do, Gabe :D.
> > > > > > > >
> > > > > > > > More seriously, a couple of points:
> > > > > > > > - Branches won't be "unseen." Their development and code
> review
> > > > will
> > > > > > > still
> > > > > > > > happen on Gerrit. It just won't spam the list on every
> commit.
> > > > > > > > - People already use this excuse not to make refactors. Or,
> > (IMO
> > > > much
> > > > > > > > worse) they just give up and don't merge their changes.
> > > > > > > >
> > > > > > > > Lately, I've noticed we've been *very* slow to incorporate
> new
> > > > > > features.
> > > > > > > > Other than systemc and the fastmodel, the only major new
> > feature
> > > in
> > > > > the
> > > > > > > > past couple of years I can think of is the ARM SVE changes,
> > which
> > > > > > > required
> > > > > > > > a herculean effort and would have been much easier with
> > > branches, I
> > > > > > > > believe. Some contributors have the resources to spend 100%
> of
> > > > their
> > > > > > time
> > > > > > > > on gem5, but that's a rare case in industry and in academia.
> We
> > > > have
> > > > > to
> > > > > > > > assume that most things will be developed by one or two
> people,
> > > > over
> > > > > > the
> > > > > > > > course of years, spending less than 50% of their time on it.
> > > > > > > >
> > > > > > > > As a specific example, I know it would be easier for me and
> my
> > > > > students
> > > > > > > to
> > > > > > > > upstream the flexcpu model we've created if we could just
> > merge a
> > > > > > branch
> > > > > > > > (with the appropriate reviews, of course!). This has
> > *required* a
> > > > lot
> > > > > > of
> > > > > > > > out of mainline development. It's so far been almost two
> years
> > > and
> > > > at
> > > > > > > least
> > > > > > > > three different students working on it to get it to a state
> > where
> > > > > it's
> > > > > > > 90%
> > > > > > > > working. It doesn't make sense to submit a patch with a half
> > > > working
> > > > > > CPU
> > > > > > > > model. As it is, we've been spending a huge amount of time
> > > rebasing
> > > > > and
> > > > > > > > re-writing history to try to get the commits to make sense
> when
> > > we
> > > > do
> > > > > > > > finally push it. It would be easier if we could just
> > incorporate
> > > > > > changes
> > > > > > > > from mainline by adding new changesets on top of what we've
> > done
> > > > > > instead
> > > > > > > of
> > > > > > > > having to rebase. This is also a good example of where I'd
> have
> > > > loved
> > > > > > to
> > > > > > > > have a stable API. In the 1.5-2 years we've been working on
> it,
> > > the
> > > > > > > > ExecContext interface has changed at least three times
> causing
> > us
> > > > to
> > > > > > have
> > > > > > > > to go through a major refactor each time. I'm certain we're
> not
> > > the
> > > > > > only
> > > > > > > > group going through this pain.
> > > > > > > >
> > > > > > > > Another use case is that most contributors want to wait
> until a
> > > > paper
> > > > > > is
> > > > > > > > published before pushing their work. Developing in the open
> one
> > > > > > changeset
> > > > > > > > at a time just isn't the most common use case for gem5.
> > > > > > > >
> > > > > > > > I agree with Gabe, it seems like gem5 is taking a way to get
> > > > better
> > > > > > but
> > > > > > > it
> > > > > > > > > will very hard for PhD students to suddenly adapt and
> review
> > > > > thousand
> > > > > > > > lines
> > > > > > > > > of code with months of yet unseen history and iteration and
> > the
> > > > > > reason
> > > > > > > > > behind every change.
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks for the feedback, Abhishek!
> > > > > > > >
> > > > > > > > Could you describe this in more detail? I'm not sure I
> > understand
> > > > > your
> > > > > > > > concern. I'll go ahead and describe what I see as the new
> flow
> > > > below,
> > > > > > > > though.
> > > > > > > >
> > > > > > > > 1. On a minor release:
> > > > > > > > You'll have a choice, you can either merge the new minor
> > release
> > > > into
> > > > > > > your
> > > > > > > > code or stay on the old release. The only reason you would
> want
> > > to
> > > > > > merge
> > > > > > > in
> > > > > > > > the new minor release is that there is a new feature or a
> > change
> > > in
> > > > > > > > behavior that you want to take advantage of. Note: we will
> make
> > > it
> > > > > very
> > > > > > > > clear in the release notes what is changed each time.
> > > > > > > >
> > > > > > > > If you choose to merge in the new minor release, *it should
> be
> > > very
> > > > > > > easy*.
> > > > > > > > We will not break any interfaces, therefore the merge should
> be
> > > > > simple.
> > > > > > > As
> > > > > > > > long as you only used "official" APIs, your code should work
> > > > *without
> > > > > > any
> > > > > > > > changes*.
> > > > > > > >
> > > > > > > > 2. On a major release
> > > > > > > > Again, you'll have the choice to merge or not. If there
> aren't
> > > any
> > > > > new
> > > > > > > > features you need, don't merge! If there are new features,
> then
> > > you
> > > > > may
> > > > > > > > want to take the effort to merge.
> > > > > > > >
> > > > > > > > To merge at a major release, you'll probably have to update
> > your
> > > > code
> > > > > > > since
> > > > > > > > APIs may have changed. These will be very well documented in
> > the
> > > > > > release
> > > > > > > > notes with information on how to update your code.
> > Additionally,
> > > > > > assuming
> > > > > > > > that you pulled the previous minor releases, you will have
> > known
> > > > this
> > > > > > was
> > > > > > > > coming because any APIs that have changed would have been
> > marked
> > > as
> > > > > > > > deprecated and produced compile time or run time warnings.
> > > > > > > >
> > > > > > > > 3. Contributing your code back to gem5
> > > > > > > > If you have a big new feature (e.g., a new CPU model), you
> can
> > > > email
> > > > > > > > gem5-dev and request a new branch. We'll set that up and then
> > we
> > > > can
> > > > > > > start
> > > > > > > > reviewing the code! There will likely be some refactoring
> > that's
> > > > > > required
> > > > > > > > or merging current development changes. You can do this *on
> > top*
> > > of
> > > > > > your
> > > > > > > > current changes, instead of having to rebase 50 changesets
> :).
> > > Once
> > > > > the
> > > > > > > > code is to a point that the maintainer and the community are
> > > happy,
> > > > > > then
> > > > > > > it
> > > > > > > > will be a simple merge commit into the develop branch!
> > > > > > > >
> > > > > > > > Of course, small bugfixes should just be on top of develop.
> > And,
> > > if
> > > > > > your
> > > > > > > > big new feature requires API changes, then it may be more
> > effort
> > > to
> > > > > get
> > > > > > > the
> > > > > > > > branch merged.
> > > > > > > >
> > > > > > > > If this doesn't make it clear, please let me know!
> > > > > > > >
> > > > > > > > ------------
> > > > > > > >
> > > > > > > > To conclude: I'm trying to build consensus here. I want to
> see
> > > gem5
> > > > > > used
> > > > > > > as
> > > > > > > > broadly as possible; I want to see gem5 used to produce
> > > > sustainable,
> > > > > > > > reproducible, and scientifically rigorous research; and I
> want
> > to
> > > > see
> > > > > > > gem5
> > > > > > > > be easy to use. I believe this new release model will help
> with
> > > > these
> > > > > > > > things, but I'm 100% open to modifications.
> > > > > > > >
> > > > > > > > I would appreciate specific actionable feedback. Simply
> saying
> > "I
> > > > > don't
> > > > > > > > like the idea" is helpful, but it would be more helpful to
> > > provide
> > > > > > > updates
> > > > > > > > the proposal with your ideas on how to accomplish the goals
> of
> > > the
> > > > > gem5
> > > > > > > > community project (again, see the Governance document:
> > > > > > > > http://gem5.org/Governance#Philosophy).
> > > > > > > >
> > > > > > > > Cheers,
> > > > > > > > Jason
> > > > > > > >
> > > > > > > > On Wed, Dec 11, 2019 at 7:49 PM Abhishek Singh <
> > > > > > > > abhishek.singh199...@gmail.com> wrote:
> > > > > > > > >
> > > > > > > > > Hi Jason,
> > > > > > > > > I agree with Gabe, it seems like gem5 is taking a way to
> get
> > > > > better
> > > > > > > but
> > > > > > > > it
> > > > > > > > > will very hard for PhD students to suddenly adapt and
> review
> > > > > thousand
> > > > > > > > lines
> > > > > > > > > of code with months of yet unseen history and iteration and
> > the
> > > > > > reason
> > > > > > > > > behind every change.
> > > > > > > > >
> > > > > > > > > And in the end it may end up to be used as a course project
> > > > rather
> > > > > > than
> > > > > > > > > thesis project or project for architecture conferences as
> > > people
> > > > > will
> > > > > > > use
> > > > > > > > > the commit or the branch which they are most familiar with.
> > > > > > > > > With so less releases every year, personally it will take a
> > lot
> > > > of
> > > > > > time
> > > > > > > > to
> > > > > > > > > relearn and understand the implementation in detail.
> > > > > > > > >
> > > > > > > > > But as this fully controlled by you, we can only suggest
> you.
> > > It
> > > > > > should
> > > > > > > > not
> > > > > > > > > be like people are creating another public gem5 as we have
> > now
> > > > and
> > > > > > use
> > > > > > > > that.
> > > > > > > > >
> > > > > > > > > This issue was raised by Gabe on Nov 27 email and today
> also,
> > > so
> > > > I
> > > > > > hope
> > > > > > > > we
> > > > > > > > > take that into account.
> > > > > > > > > It feels like gem5 is getting towards privatization...
> > > > > > > > >
> > > > > > > > > It’s just a opinion, it may be wrong.
> > > > > > > > >
> > > > > > > > > On Wed, Dec 11, 2019 at 10:12 PM Gabe Black <
> > > > gabebl...@google.com>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Could you please clarify the policy on breaking APIs? It
> > > makes
> > > > > > sense
> > > > > > > > for
> > > > > > > > > > releases to maintain stable APIs, but how does that apply
> > to
> > > > the
> > > > > > > > > > development branch? I'm worried that it will be very hard
> > to
> > > > make
> > > > > > > > changes
> > > > > > > > > > that don't change any interfaces, and we definitely don't
> > > want
> > > > to
> > > > > > > > encourage
> > > > > > > > > > a style of development where we just add and add and add
> > > > without
> > > > > > ever
> > > > > > > > > > refactoring or consolidating things. If APIs can continue
> > to
> > > > > change
> > > > > > > as
> > > > > > > > > > needed in the development branch and we just need to warn
> > > > people
> > > > > > > before
> > > > > > > > > > they're released, that seems reasonable.
> > > > > > > > > >
> > > > > > > > > > In general (and not necessarily tied to this proposal),
> we
> > > > should
> > > > > > try
> > > > > > > > very
> > > > > > > > > > hard not to have silos where people develop incompatible
> > > > features
> > > > > > and
> > > > > > > > don't
> > > > > > > > > > add to gem5 in a holistic way. It's easiest to wall off
> > > > > > functionality
> > > > > > > > > > behind ifdefs or config options or in separate
> repositories
> > > and
> > > > > > then
> > > > > > > > just
> > > > > > > > > > ignore it when implementing new features, but that
> creates
> > a
> > > > lot
> > > > > of
> > > > > > > > > > technical debt and complexity which really cripples both
> > gem5
> > > > and
> > > > > > > > future
> > > > > > > > > > development and compounds exponentially over time.
> > > > > > > > > >
> > > > > > > > > > Back to the branches, I'm still not a fan. I think it
> gives
> > > > > people
> > > > > > an
> > > > > > > > > > excuse not to make changes ("you want me to refactor 18
> > > months
> > > > of
> > > > > > > code
> > > > > > > > for
> > > > > > > > > > that?" (not an actual quote)), on top of other issues.
> Also
> > > > there
> > > > > > are
> > > > > > > > more
> > > > > > > > > > than two options. You can do your work on top of tree
> like
> > I
> > > > do,
> > > > > > even
> > > > > > > > when
> > > > > > > > > > adding big new features. It ensures that you don't get
> too
> > > > > invested
> > > > > > > in
> > > > > > > > bad
> > > > > > > > > > paths and don't have a mountain of integration work to do
> > > when
> > > > > you
> > > > > > > > finally
> > > > > > > > > > decide to bridge back in. It also gives the community an
> > > > > > appropriate
> > > > > > > > > > opportunity to provide feedback through review which is
> not
> > > > > > practical
> > > > > > > > when
> > > > > > > > > > dealing with dozens of files and thousands of lines of
> code
> > > > with
> > > > > > > months
> > > > > > > > of
> > > > > > > > > > yet unseen history and iteration.
> > > > > > > > > >
> > > > > > > > > > Gabe
> > > > > > > > > >
> > > > > > > > > > On Wed, Dec 11, 2019 at 5:30 PM Jason Lowe-Power <
> > > > > > > ja...@lowepower.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi all,
> > > > > > > > > > >
> > > > > > > > > > > Here are the specific changes we're proposing. Please
> > give
> > > us
> > > > > > your
> > > > > > > > > > > feedback! If you disagree with some part of this
> > proposal,
> > > > let
> > > > > us
> > > > > > > > know
> > > > > > > > > > > why and what you suggest instead.
> > > > > > > > > > >
> > > > > > > > > > > # What's changing now
> > > > > > > > > > >
> > > > > > > > > > > 1) We are going to release gem5 19 "soon" (two blocking
> > > > issues:
> > > > > > 1)
> > > > > > > > > > > this proposal and 2) the website)
> > > > > > > > > > > 2) When we release gem5 19 we are going to create a new
> > > > branch
> > > > > > > called
> > > > > > > > > > > "develop" which branches at the point we make the
> > release.
> > > > > > > > > > > 3) We are going to tag master with "19.0.0-beta"
> > > > > > > > > > >
> > > > > > > > > > > After this, all development will move to the develop
> > branch
> > > > > > instead
> > > > > > > > of
> > > > > > > > > > > master. This will allow the common case of people using
> > > gem5
> > > > > for
> > > > > > > > > > > research to be the default. When cloning gem5 you will
> > get
> > > a
> > > > > > > "stable"
> > > > > > > > > > > release.
> > > > > > > > > > >
> > > > > > > > > > > ## Changes for developers
> > > > > > > > > > >
> > > > > > > > > > > There will be two changes for developers:
> > > > > > > > > > > 1) You'll have to checkout develop after cloning
> > > > > > > > > > > 2) You'll have to push to "refs/for/develop" instead of
> > > > > > > > "refs/for/master"
> > > > > > > > > > >
> > > > > > > > > > > # Next release
> > > > > > > > > > >
> > > > > > > > > > > When we get to our next release (gem5 20, yay!), we
> will
> > > > merge
> > > > > > the
> > > > > > > > > > > develop branch into master. Assuming no hotfixes, this
> > will
> > > > be
> > > > > a
> > > > > > > > > > > simple fast forward.
> > > > > > > > > > >
> > > > > > > > > > > We will have a major release once per year usually in
> > April
> > > > > > (after
> > > > > > > > the
> > > > > > > > > > > MICRO deadline but with enough lead time to put
> together
> > an
> > > > > ISCA
> > > > > > > > > > > tutorial). We will have two minor releases later in the
> > > year,
> > > > > one
> > > > > > > in
> > > > > > > > > > > August and one in December.
> > > > > > > > > > >
> > > > > > > > > > > # Why are we going to releases?
> > > > > > > > > > >
> > > > > > > > > > > It's important for our *users* (remember, we're
> > developing
> > > > gem5
> > > > > > to
> > > > > > > be
> > > > > > > > > > > used by other people, not just core developers!) to be
> > able
> > > > to
> > > > > > > build
> > > > > > > > > > > off of stable APIs. Therefore, between major releases
> *we
> > > > will
> > > > > > not
> > > > > > > > > > > change any APIs*. Code that we expect other people will
> > > build
> > > > > off
> > > > > > > of
> > > > > > > > > > > are APIs. This includes, but is not limited to:
> > > > > > > > > > > - Ports
> > > > > > > > > > > - ExecContext
> > > > > > > > > > > - Packet
> > > > > > > > > > > - SimObject, ClockedObject, etc.
> > > > > > > > > > > - Event, EventQueue, etc.
> > > > > > > > > > > - Command line options
> > > > > > > > > > > - And probably many more. We will work on finalizing
> this
> > > > over
> > > > > > the
> > > > > > > > > > > next couple of months before the gem5-20 release.
> > > > > > > > > > >
> > > > > > > > > > > ## What does going to releases mean for developers?
> > > > > > > > > > >
> > > > > > > > > > > Thus, if any developers want to make changes to the
> above
> > > > code,
> > > > > > we
> > > > > > > > must
> > > > > > > > > > > Mark the API as "deprecated" at least by the *.2
> December
> > > > > release
> > > > > > > to
> > > > > > > > > > > give users at least one quarter to prepare.
> > > > > > > > > > > AND either
> > > > > > > > > > > 1) Make backwards compatible changes (e.g., just add a
> > new
> > > > > > function
> > > > > > > > to
> > > > > > > > > > > the API that we will switch to when the API merge
> window
> > > > opens
> > > > > > > (after
> > > > > > > > > > > the December minor release))
> > > > > > > > > > > or 2) Create a new branch from develop that we wait to
> > > merge
> > > > > > until
> > > > > > > > the
> > > > > > > > > > > merge window.
> > > > > > > > > > >
> > > > > > > > > > > # Feature branches
> > > > > > > > > > >
> > > > > > > > > > > Despite the disagreement in this thread so far, we're
> > still
> > > > > > > proposing
> > > > > > > > > > > feature branches. However, they should be *rare*, and
> > only
> > > > > > > > maintainers
> > > > > > > > > > > will have the ability to add new branches. The exact
> > gerrit
> > > > > > > > mechanisms
> > > > > > > > > > > for this need to be figured out.
> > > > > > > > > > >
> > > > > > > > > > > We *require* something like feature branches given this
> > new
> > > > > > release
> > > > > > > > > > > model for two reasons:
> > > > > > > > > > > 1) There will be API breaking changes that will be
> > > long-lived
> > > > > (at
> > > > > > > > > > > least between a major release and the next API change
> > merge
> > > > > > > window).
> > > > > > > > > > > 2) Many people develop new features to incorporate into
> > > gem5
> > > > > and
> > > > > > > > > > > frequently they take months of *iteration* before they
> > are
> > > > > ready
> > > > > > to
> > > > > > > > be
> > > > > > > > > > > merged. AMD's GPU model in the gcn-staging branch is
> one
> > > > > example
> > > > > > > > > > > (
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://gem5.googlesource.com/amd/gem5/+/refs/heads/agutierr/master-gcn3-staging
> > > > > > > > > > > ),
> > > > > > > > > > > the FlexCPU model my group has been working on is
> another
> > > > > > > > > > > (https://github.com/darchr/gem5/tree/flexcpu).
> > > > > > > > > > > We need an improved method of merging these kinds of
> > > changes.
> > > > > > > > > > >
> > > > > > > > > > > The way I see it, we have two options: continue
> > development
> > > > > > locally
> > > > > > > > in
> > > > > > > > > > > our own repos as we are doing right now. OR we could
> move
> > > the
> > > > > > > > > > > development *into the open* by creating feature
> branches
> > on
> > > > the
> > > > > > > main
> > > > > > > > > > > gem5 repo. I prefer the latter. There's little
> difference
> > > to
> > > > > > > Gerrit.
> > > > > > > > > > > The main difference is what you get when you do git
> pull.
> > > > > > > > > > >
> > > > > > > > > > > It will be up to the maintainers to make sure branches
> > are
> > > > > merged
> > > > > > > in
> > > > > > > > a
> > > > > > > > > > > timely fashion and we don't create too many branches.
> > They
> > > > > should
> > > > > > > be
> > > > > > > > > > > kept up to date by their maintainer by frequently
> merging
> > > the
> > > > > > > develop
> > > > > > > > > > > branch.
> > > > > > > > > > >
> > > > > > > > > > > # Hotfixes to the stable/master branch
> > > > > > > > > > >
> > > > > > > > > > > It's likely we'll find bugs while developing gem5. If
> the
> > > > > > > maintainer
> > > > > > > > > > > for the component thinks the bug is severe enough (I
> > don't
> > > > want
> > > > > > to
> > > > > > > > > > > make any specific guidelines right now), we will
> backport
> > > > that
> > > > > > bug
> > > > > > > to
> > > > > > > > > > > the master branch. We'll have to work on our process
> for
> > > this
> > > > > to
> > > > > > > > > > > ensure that develop will cleanly merge into master
> later,
> > > but
> > > > > > this
> > > > > > > > > > > should be straightforward.
> > > > > > > > > > >
> > > > > > > > > > > # Feedback
> > > > > > > > > > >
> > > > > > > > > > > Please let us know what you think! We're going to go
> one
> > > more
> > > > > > round
> > > > > > > > of
> > > > > > > > > > > feedback on the gem5-dev list, then we'll post the
> > > finalized
> > > > > > > version
> > > > > > > > > > > on gem5-users for a last round of feedback. I'm hoping
> to
> > > > begin
> > > > > > to
> > > > > > > > > > > implement this early next year!
> > > > > > > > > > >
> > > > > > > > > > > Thanks for your patience on this. I know it's a lot to
> > read
> > > > > > through
> > > > > > > > > > > and think about, but it's important that we get this
> > > process
> > > > > > right
> > > > > > > > the
> > > > > > > > > > > first time. We don't want to have to change this every
> > year
> > > > :).
> > > > > > > > > > >
> > > > > > > > > > > Cheers,
> > > > > > > > > > > Jason
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Dec 10, 2019 at 4:49 PM Bobby Bruce <
> > > > > bbr...@ucdavis.edu>
> > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Thank you all for your comments on this.
> > > > > > > > > > > >
> > > > > > > > > > > > In regards to the master/develop branch idea, I'm a
> big
> > > > > > advocate
> > > > > > > of
> > > > > > > > > > > Jason's
> > > > > > > > > > > > "make the most common use the easiest" philosophy. I
> > > think
> > > > he
> > > > > > > did a
> > > > > > > > > > good
> > > > > > > > > > > > job of describing why this is important. We use the
> git
> > > > repo
> > > > > as
> > > > > > > our
> > > > > > > > > > > > primarily way of distributing gem5, and, as such, I
> > > believe
> > > > > the
> > > > > > > > default
> > > > > > > > > > > > behaviour should be that when a user clones the repo,
> > > they
> > > > > are
> > > > > > > > getting
> > > > > > > > > > > the
> > > > > > > > > > > > latest stable release to build on top of. I strongly
> > > > suspect
> > > > > > > users
> > > > > > > > will
> > > > > > > > > > > > disproportionately use whatever `git clone` gives
> them
> > > > rather
> > > > > > > than
> > > > > > > > move
> > > > > > > > > > > > between branches or tags in order to get specific,
> > stable
> > > > > > > releases.
> > > > > > > > If
> > > > > > > > > > > most
> > > > > > > > > > > > users end up working on the very latest "unstable"
> > > > > development
> > > > > > > > version
> > > > > > > > > > > then
> > > > > > > > > > > > our release system becomes pointless. I realize as
> > > everyone
> > > > > on
> > > > > > > this
> > > > > > > > > > list
> > > > > > > > > > > is
> > > > > > > > > > > > a gem5 contributor, there is a preference for the
> > > > development
> > > > > > > > branch to
> > > > > > > > > > > be
> > > > > > > > > > > > default, but I'd hope we are in the minority of
> people
> > > who
> > > > > > > actually
> > > > > > > > use
> > > > > > > > > > > > gem5. My vote therefore still goes to having two
> > > branches:
> > > > > the
> > > > > > > > > > > > master/"release" branch and the "develop" branch for
> > > > > day-to-day
> > > > > > > > > > > > development. Hotfixes would be created on a separate
> > > branch
> > > > > and
> > > > > > > > merged
> > > > > > > > > > > into
> > > > > > > > > > > > both upon completion.
> > > > > > > > > > > >
> > > > > > > > > > > > Regarding feature branching --- My intention with
> this
> > > part
> > > > > of
> > > > > > > the
> > > > > > > > > > > proposal
> > > > > > > > > > > > was not for feature branches to be the default manner
> > in
> > > > > which
> > > > > > > > people
> > > > > > > > > > > > commit to gem5 (as they would be if we were to
> > implement
> > > > the
> > > > > > > > GitFlow
> > > > > > > > > > > model
> > > > > > > > > > > > in full), I consider them something we should allow,
> in
> > > > some
> > > > > > > form,
> > > > > > > > to
> > > > > > > > > > > > improve the development process in certain
> > circumstances,
> > > > but
> > > > > > > that
> > > > > > > > they
> > > > > > > > > > > > should be a rather rare thing. For example, a new
> > > component
> > > > > may
> > > > > > > > need
> > > > > > > > > > > > several months of development work, consisting of
> many
> > > > > commits.
> > > > > > > > > > > Developers,
> > > > > > > > > > > > at present, have two options: either A) gradually
> > > > incorporate
> > > > > > > > commits
> > > > > > > > > > > over
> > > > > > > > > > > > the months on the master branch, or B) push all the
> > > commits
> > > > > to
> > > > > > > the
> > > > > > > > > > master
> > > > > > > > > > > > branch in one go. I don't think either are ideal
> > > > solutions.
> > > > > A
> > > > > > > > > > dedicated
> > > > > > > > > > > > feature branch would quarantine these changes while
> in
> > > > active
> > > > > > > > > > > development,
> > > > > > > > > > > > to be merged at a later date. A second circumstance
> > that
> > > > > could
> > > > > > > > arise is
> > > > > > > > > > > > when we wish to postpone the incorporation of a
> > feature.
> > > A
> > > > > > change
> > > > > > > > that
> > > > > > > > > > > > requires an API alteration, for example, should be
> > > held-off
> > > > > > until
> > > > > > > > we
> > > > > > > > > > are
> > > > > > > > > > > > ready to deploy a major release. Isolating such a
> > change
> > > > in a
> > > > > > > > feature
> > > > > > > > > > > > branch, to be merged later, may be necessary. This
> all
> > > > being
> > > > > > > said,
> > > > > > > > I
> > > > > > > > > > > > acknowledge there is a risk in code getting lost
> > > > > indefinitely,
> > > > > > or
> > > > > > > > for
> > > > > > > > > > > long
> > > > > > > > > > > > periods of time, in feature branches, and our repo
> > > getting
> > > > > > messy
> > > > > > > as
> > > > > > > > a
> > > > > > > > > > > > result. After some discussion with Jason, I believe
> the
> > > > best
> > > > > of
> > > > > > > > both
> > > > > > > > > > > worlds
> > > > > > > > > > > > can be achieved if we make feature branches "by
> > request"
> > > > ---
> > > > > > > anyone
> > > > > > > > > > > wanting
> > > > > > > > > > > > to create a feature branch would need to ask a
> > maintainer
> > > > > (or a
> > > > > > > > PCM) to
> > > > > > > > > > > > create one for them. Whether or not a feature branch
> is
> > > to
> > > > be
> > > > > > > > created
> > > > > > > > > > > would
> > > > > > > > > > > > be decided upon on a case-by-case basis.
> > > > > > > > > > > >
> > > > > > > > > > > > I hope no one feels like I'm dictating anything here.
> > I'm
> > > > > > writing
> > > > > > > > this
> > > > > > > > > > > > primarily to prompt more discussion, so please feel
> > free
> > > to
> > > > > > > counter
> > > > > > > > any
> > > > > > > > > > > of
> > > > > > > > > > > > my points or suggest alternatives. Simple messages of
> > > > > support,
> > > > > > as
> > > > > > > > well
> > > > > > > > > > as
> > > > > > > > > > > > criticism, would also be appreciated to help us get a
> > > feel
> > > > > for
> > > > > > > what
> > > > > > > > is,
> > > > > > > > > > > or
> > > > > > > > > > > > what is not, proving popular with the community.
> > > > > > > > > > > >
> > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > Bobby
> > > > > > > > > > > > --
> > > > > > > > > > > > Dr. Bobby R. Bruce
> > > > > > > > > > > > Room 2235,
> > > > > > > > > > > > Kemper Hall, UC Davis
> > > > > > > > > > > > Davis,
> > > > > > > > > > > > CA, 95616
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, Nov 27, 2019 at 4:49 PM Gabe Black <
> > > > > > gabebl...@google.com
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > The idea of feature branches makes me very nervous.
> > It
> > > > > sounds
> > > > > > > > like a
> > > > > > > > > > > dark
> > > > > > > > > > > > > corner for a large body of code to grow in, and
> then
> > > once
> > > > > it
> > > > > > > pops
> > > > > > > > > > back
> > > > > > > > > > > into
> > > > > > > > > > > > > the light there may be very ingrained and difficult
> > to
> > > > fix
> > > > > > > > problems
> > > > > > > > > > > which
> > > > > > > > > > > > > either have to be ignored or fixed. A long standing
> > > > branch
> > > > > > may
> > > > > > > > also
> > > > > > > > > > be
> > > > > > > > > > > very
> > > > > > > > > > > > > hard to reconcile with ToT, and it may be very
> > tempting
> > > > to
> > > > > > just
> > > > > > > > let
> > > > > > > > > > it
> > > > > > > > > > > die
> > > > > > > > > > > > > rather than try to jump that energy barrier and get
> > it
> > > > back
> > > > > > > into
> > > > > > > > > > > mainline.
> > > > > > > > > > > > >
> > > > > > > > > > > > > This would basically be sanctifying the practice of
> > > > > > > perma-forking
> > > > > > > > > > gem5,
> > > > > > > > > > > > > which unfortunately happens anyway now and then.
> > There
> > > > have
> > > > > > > been
> > > > > > > > a
> > > > > > > > > > > number
> > > > > > > > > > > > > of times on the mailing list someone says something
> > > along
> > > > > the
> > > > > > > > lines
> > > > > > > > > > of
> > > > > > > > > > > "I
> > > > > > > > > > > > > downloaded special version of gem5 foo from some
> > > github"
> > > > > and
> > > > > > it
> > > > > > > > turns
> > > > > > > > > > > out
> > > > > > > > > > > > > it's X years old with a bunch of old and new bugs,
> > > nobody
> > > > > > knows
> > > > > > > > how
> > > > > > > > > > it
> > > > > > > > > > > > > works, etc. Let's not encourage that :-).
> > > > > > > > > > > > >
> > > > > > > > > > > > > Gabe
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Wed, Nov 27, 2019 at 9:17 AM Vince Harron <
> > > > > > > vhar...@google.com
> > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > I agree with Ciro that the de facto standard used
> > by
> > > > most
> > > > > > > > projects
> > > > > > > > > > is
> > > > > > > > > > > > > that
> > > > > > > > > > > > > > mainline development should happen in master and
> > > > releases
> > > > > > > > should be
> > > > > > > > > > > new
> > > > > > > > > > > > > > branches created off of master.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > You could also create a branch “stable” or
> “latest”
> > > > that
> > > > > > gets
> > > > > > > > > > > updated to
> > > > > > > > > > > > > > the latest release.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The default download instructions could say
> > something
> > > > > like
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > git clone <url> —branch stable
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Wed, Nov 27, 2019 at 8:54 AM Jason Lowe-Power
> <
> > > > > > > > > > > ja...@lowepower.com>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Thanks for the input Ciro!
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > My pushback here is that most people who use
> gem5
> > > > will
> > > > > be
> > > > > > > > making
> > > > > > > > > > > > > > > modifications to it (at least for the
> foreseeable
> > > > > > future).
> > > > > > > I
> > > > > > > > > > don't
> > > > > > > > > > > > > think
> > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > want to common use case to be 1) Download a
> > > tarball,
> > > > 2)
> > > > > > > make
> > > > > > > > > > local
> > > > > > > > > > > > > > > modifications to the source. I think we want
> > people
> > > > to
> > > > > 1)
> > > > > > > > clone a
> > > > > > > > > > > > > > "stable"
> > > > > > > > > > > > > > > gem5, 2) make modifications to the stable
> branch
> > > for
> > > > > > their
> > > > > > > > > > > research *in
> > > > > > > > > > > > > > > git*.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > My experience with architecture grad students
> and
> > > > > > > researchers
> > > > > > > > is
> > > > > > > > > > > that
> > > > > > > > > > > > > if
> > > > > > > > > > > > > > > you don't force them to use things like git
> most
> > of
> > > > > them
> > > > > > > > won't.
> > > > > > > > > > > One of
> > > > > > > > > > > > > my
> > > > > > > > > > > > > > > (personal) goals is to make it easy/default to
> > use
> > > > gem5
> > > > > > > "the
> > > > > > > > > > right
> > > > > > > > > > > > > way".
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > In fact, I almost don't want to provide
> tarballs
> > at
> > > > > all.
> > > > > > > > Unless
> > > > > > > > > > we
> > > > > > > > > > > can
> > > > > > > > > > > > > > > provide pre-build gem5 binaries, I don't think
> it
> > > > makes
> > > > > > > sense
> > > > > > > > to
> > > > > > > > > > > > > provide
> > > > > > > > > > > > > > > tarballs. I don't think prebuilt binaries make
> > > sense
> > > > > for
> > > > > > > the
> > > > > > > > > > > research
> > > > > > > > > > > > > use
> > > > > > > > > > > > > > > case of gem5 since almost all research will
> > require
> > > > > > > extending
> > > > > > > > > > gem5
> > > > > > > > > > > and
> > > > > > > > > > > > > > > there's no way to dynamically load in new
> modules
> > > > > (though
> > > > > > > we
> > > > > > > > > > should
> > > > > > > > > > > > > > support
> > > > > > > > > > > > > > > this in the future!). For teaching with gem5
> > > prebuilt
> > > > > > > > binaries
> > > > > > > > > > > could
> > > > > > > > > > > > > make
> > > > > > > > > > > > > > > sense, but I think we'll hold off on that for
> > now,
> > > > TBH.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I think this comes down to "make the common
> case
> > > > > fast". I
> > > > > > > > believe
> > > > > > > > > > > the
> > > > > > > > > > > > > > > common case is researchers who want to make
> minor
> > > > > changes
> > > > > > > to
> > > > > > > > gem5
> > > > > > > > > > > > > (e.g.,
> > > > > > > > > > > > > > > add a new SimObject or modify the cache
> protocol)
> > > and
> > > > > run
> > > > > > > > > > > experiments
> > > > > > > > > > > > > for
> > > > > > > > > > > > > > > their research. Developing gem5 for the
> community
> > > > > (using
> > > > > > > the
> > > > > > > > > > > > > development
> > > > > > > > > > > > > > > branch) is actually the uncommon case.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > BTW, I agree with you that we should try to
> > follow
> > > > > > whatever
> > > > > > > > > > > practices
> > > > > > > > > > > > > are
> > > > > > > > > > > > > > > most popular when it makes sense. However, in
> > many
> > > > ways
> > > > > > > gem5
> > > > > > > > is a
> > > > > > > > > > > > > unique
> > > > > > > > > > > > > > > project, so following others doesn't always
> make
> > > > sense
> > > > > > :).
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I'm open to discussion on this. Thoughts?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Cheers,
> > > > > > > > > > > > > > > Jason
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Wed, Nov 27, 2019 at 12:01 AM Ciro Santilli
> <
> > > > > > > > > > > ciro.santi...@arm.com>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Thanks Bobby for pushing this forward, having
> > > > > releases
> > > > > > > > would
> > > > > > > > > > be a
> > > > > > > > > > > > > good
> > > > > > > > > > > > > > > > thing for gem5.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I would recommend against having both master
> > and
> > > > > > develop
> > > > > > > > branch
> > > > > > > > > > > > > though,
> > > > > > > > > > > > > > > > because in the large majority of projects out
> > > > there,
> > > > > > > master
> > > > > > > > ==
> > > > > > > > > > > the
> > > > > > > > > > > > > > latest
> > > > > > > > > > > > > > > > version, so I'm always confused when I have
> to
> > > > switch
> > > > > > to
> > > > > > > > > > develop,
> > > > > > > > > > > > > > > specially
> > > > > > > > > > > > > > > > if it isn't the default HEAD (is is however
> > > > possible
> > > > > to
> > > > > > > set
> > > > > > > > the
> > > > > > > > > > > > > default
> > > > > > > > > > > > > > > > branch to develop as shown as:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://gerrit-review.googlesource.com/Documentation/project-configuration.html#default-branch
> > > > > > > > > > > > > > > > )
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > This is even more important if we start
> having
> > > > > prebuilt
> > > > > > > > > > releases
> > > > > > > > > > > or
> > > > > > > > > > > > > > tgzs
> > > > > > > > > > > > > > > > with source (which are smaller than full
> repo)
> > > for
> > > > > the
> > > > > > > > tags, at
> > > > > > > > > > > which
> > > > > > > > > > > > > > > point
> > > > > > > > > > > > > > > > basically everyone who clones wants the
> latest.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I would instead recommend:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > - master: latest version
> > > > > > > > > > > > > > > > - v2019: a tag. The latest one can be easily
> > > found
> > > > > > with:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://stackoverflow.com/questions/1404796/how-to-get-the-latest-tag-name-in-current-branch-in-git
> > > > > > > > > > > > > > > > - 2019 or b2019: branch created at the same
> > time
> > > as
> > > > > > v2019
> > > > > > > > to
> > > > > > > > > > > which
> > > > > > > > > > > > > > > > backports are applied, and to which v2019.0.1
> > > tags
> > > > > can
> > > > > > > also
> > > > > > > > be
> > > > > > > > > > > > > > optionally
> > > > > > > > > > > > > > > > applied
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > ________________________________
> > > > > > > > > > > > > > > > From: gem5-dev <gem5-dev-boun...@gem5.org>
> on
> > > > behalf
> > > > > > of
> > > > > > > > Bobby
> > > > > > > > > > > Bruce
> > > > > > > > > > > > > <
> > > > > > > > > > > > > > > > bbr...@ucdavis.edu>
> > > > > > > > > > > > > > > > Sent: Wednesday, November 27, 2019 12:10 AM
> > > > > > > > > > > > > > > > To: gem5-dev@gem5.org <gem5-dev@gem5.org>
> > > > > > > > > > > > > > > > Subject: [gem5-dev] gem5 19.0.0 : New git
> > > branching
> > > > > > > > proposal
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Dear all,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > As you know, at the end of this quarter we
> will
> > > be
> > > > > > > > releasing
> > > > > > > > > > > gem5-19,
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > first official release of gem5. As part of
> this
> > > > > release
> > > > > > > > we'd
> > > > > > > > > > > like to
> > > > > > > > > > > > > > > change
> > > > > > > > > > > > > > > > our git branching structure. Therefore, I'm
> > > writing
> > > > > to
> > > > > > > ask
> > > > > > > > for
> > > > > > > > > > > > > feedback
> > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > what we have planned and whether it can be
> > > improved
> > > > > > upon.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > We'd like to have a git repo structure
> similar
> > to
> > > > > that
> > > > > > > used
> > > > > > > > in
> > > > > > > > > > > > > gitflow
> > > > > > > > > > > > > > > > development:
> > > > > > > > > > >
> https://nvie.com/posts/a-successful-git-branching-model
> > > > > > > > > > > > > .
> > > > > > > > > > > > > > > > I've
> > > > > > > > > > > > > > > > seen this model work well before, as it has
> > some
> > > > > > > worthwhile
> > > > > > > > > > > > > abstractors
> > > > > > > > > > > > > > > for
> > > > > > > > > > > > > > > > public, open source git projects with regular
> > > > > releases.
> > > > > > > > What
> > > > > > > > > > I'd
> > > > > > > > > > > like
> > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > incorporate from this model is the following:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > - Two permanent branches: master and develop.
> > > > > > > > > > > > > > > > - "develop" would function as master does
> now.
> > > This
> > > > > > would
> > > > > > > > be
> > > > > > > > > > the
> > > > > > > > > > > main
> > > > > > > > > > > > > > > point
> > > > > > > > > > > > > > > > in which changes are applied between gem5
> > > releases.
> > > > > > > > > > > > > > > > - Upon a new release of gem5, the develop
> > branch
> > > > > would
> > > > > > be
> > > > > > > > > > merged
> > > > > > > > > > > into
> > > > > > > > > > > > > > > > master and a new git tag added to master
> > > indicating
> > > > > the
> > > > > > > > release
> > > > > > > > > > > > > > version.
> > > > > > > > > > > > > > > > Ergo, the master branch would always contain
> > the
> > > > > latest
> > > > > > > > release
> > > > > > > > > > > of
> > > > > > > > > > > > > > gem5.
> > > > > > > > > > > > > > > > - If a quick hotfix is needed, a new "hotfix"
> > > > branch
> > > > > > > would
> > > > > > > > be
> > > > > > > > > > > created
> > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > merged into both the develop and master
> > branches
> > > > upon
> > > > > > > > > > completion.
> > > > > > > > > > > > > This
> > > > > > > > > > > > > > > > would also require a new tag on the master
> > > branch.
> > > > (I
> > > > > > > > suggest
> > > > > > > > > > > using
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > standard "Version [Major].[Minor].[Hotfix]"
> > > version
> > > > > > > > numbering
> > > > > > > > > > > system.
> > > > > > > > > > > > > > > I.e.,
> > > > > > > > > > > > > > > > the first version would be V19.0.0, a hotfix
> to
> > > > this
> > > > > > > would
> > > > > > > > make
> > > > > > > > > > > it
> > > > > > > > > > > > > > > V19.0.1,
> > > > > > > > > > > > > > > > and a minor release would make it V19.1.0).
> > > > > > > > > > > > > > > > - The creation of feature branches would be
> > > > > permitted.
> > > > > > > > These
> > > > > > > > > > > branches
> > > > > > > > > > > > > > > would
> > > > > > > > > > > > > > > > encapsulate the gradual development of large
> > > > features
> > > > > > > > (i.e.,
> > > > > > > > > > ones
> > > > > > > > > > > > > > carried
> > > > > > > > > > > > > > > > out over many commits). When complete a
> feature
> > > > > branch
> > > > > > > > would be
> > > > > > > > > > > > > merged
> > > > > > > > > > > > > > > into
> > > > > > > > > > > > > > > > the develop branch. They'd be no obligation
> to
> > > use
> > > > > > > feature
> > > > > > > > > > > branches
> > > > > > > > > > > > > > > though
> > > > > > > > > > > > > > > > we believe they could be of value in certain
> > > cases.
> > > > > For
> > > > > > > > > > example,
> > > > > > > > > > > if a
> > > > > > > > > > > > > > > > developer wishes to postpone a developed
> > feature
> > > > for
> > > > > a
> > > > > > > > given
> > > > > > > > > > gem5
> > > > > > > > > > > > > > release
> > > > > > > > > > > > > > > > (e.g., something more suited for a major
> > release
> > > > > rather
> > > > > > > > than a
> > > > > > > > > > > minor
> > > > > > > > > > > > > > > one),
> > > > > > > > > > > > > > > > then they could submit their changes as a
> > feature
> > > > > > branch
> > > > > > > > and
> > > > > > > > > > > wait to
> > > > > > > > > > > > > > > merge
> > > > > > > > > > > > > > > > to the develop branch at a later date.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I believe this setup would make our
> development
> > > > > process
> > > > > > > run
> > > > > > > > > > > smoother
> > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > give gem5 users more stability. Day-to-day
> > > > > development
> > > > > > > > wouldn't
> > > > > > > > > > > > > change
> > > > > > > > > > > > > > > much
> > > > > > > > > > > > > > > > as committing to the develop branch would
> work
> > in
> > > > the
> > > > > > > same
> > > > > > > > way
> > > > > > > > > > as
> > > > > > > > > > > > > > > > submitting to master does now.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > If anyone has any thoughts about this, I'd be
> > > happy
> > > > > to
> > > > > > > hear
> > > > > > > > > > from
> > > > > > > > > > > you.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > > > > > Bobby
> > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > Dr. Bobby R. Bruce
> > > > > > > > > > > > > > > > Room 2235,
> > > > > > > > > > > > > > > > Kemper Hall, UC Davis
> > > > > > > > > > > > > > > > Davis,
> > > > > > > > > > > > > > > > CA, 95616
> > > > > > > > > > > > > > > >
> _______________________________________________
> > > > > > > > > > > > > > > > gem5-dev mailing list
> > > > > > > > > > > > > > > > gem5-dev@gem5.org
> > > > > > > > > > > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev
> > > > > > > > > > > > > > > >
> _______________________________________________
> > > > > > > > > > > > > > > > gem5-dev mailing list
> > > > > > > > > > > > > > > > gem5-dev@gem5.org
> > > > > > > > > > > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev
> > > > > > > > > > > > > > > _______________________________________________
> > > > > > > > > > > > > > > gem5-dev mailing list
> > > > > > > > > > > > > > > gem5-dev@gem5.org
> > > > > > > > > > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Vince Harron | Engineering Manager |
> > > > > vhar...@google.com
> > > > > > |
> > > > > > > > > > > > > 858-442-0868 <(858)%20442-0868> <(858)%20442-0868>
> > <(858)%20442-0868>
> > > > > > > > > > > > > > <(858)%20442-0868>
> > > > > > > > > > > > > > _______________________________________________
> > > > > > > > > > > > > > gem5-dev mailing list
> > > > > > > > > > > > > > gem5-dev@gem5.org
> > > > > > > > > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev
> > > > > > > > > > > > > _______________________________________________
> > > > > > > > > > > > > gem5-dev mailing list
> > > > > > > > > > > > > gem5-dev@gem5.org
> > > > > > > > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev
> > > > > > > > > > > > _______________________________________________
> > > > > > > > > > > > gem5-dev mailing list
> > > > > > > > > > > > gem5-dev@gem5.org
> > > > > > > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev
> > > > > > > > > > > _______________________________________________
> > > > > > > > > > > gem5-dev mailing list
> > > > > > > > > > > gem5-dev@gem5.org
> > > > > > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev
> > > > > > > > > > _______________________________________________
> > > > > > > > > > gem5-dev mailing list
> > > > > > > > > > gem5-dev@gem5.org
> > > > > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev
> > > > > > > > > _______________________________________________
> > > > > > > > > gem5-dev mailing list
> > > > > > > > > gem5-dev@gem5.org
> > > > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev
> > > > > > > > _______________________________________________
> > > > > > > > gem5-dev mailing list
> > > > > > > > gem5-dev@gem5.org
> > > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev
> > > > > > > _______________________________________________
> > > > > > > gem5-dev mailing list
> > > > > > > gem5-dev@gem5.org
> > > > > > > http://m5sim.org/mailman/listinfo/gem5-dev
> > > > > > _______________________________________________
> > > > > > gem5-dev mailing list
> > > > > > gem5-dev@gem5.org
> > > > > > http://m5sim.org/mailman/listinfo/gem5-dev
> > > > > _______________________________________________
> > > > > gem5-dev mailing list
> > > > > gem5-dev@gem5.org
> > > > > http://m5sim.org/mailman/listinfo/gem5-dev
> > > > _______________________________________________
> > > > gem5-dev mailing list
> > > > gem5-dev@gem5.org
> > > > http://m5sim.org/mailman/listinfo/gem5-dev
> > > _______________________________________________
> > > gem5-dev mailing list
> > > gem5-dev@gem5.org
> > > http://m5sim.org/mailman/listinfo/gem5-dev
> > _______________________________________________
> > gem5-dev mailing list
> > gem5-dev@gem5.org
> > http://m5sim.org/mailman/listinfo/gem5-dev
> _______________________________________________
> gem5-dev mailing list
> gem5-dev@gem5.org
> http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient,
please notify the sender immediately and do not disclose the contents to any
other person, use it for any purpose, or store or copy the information in any
medium. Thank you.
_______________________________________________
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev
