On Fri, 7 Nov 2014, Rainer Gerhards wrote:
I have this luxury of rewriting my changes because I work in a private
branch. Since you push your changes early and often, rebasing (editing,
squashing) might be problematic - unless it is made a policy that there are
no guaranties about what happens in a feature branch.
Well, I now also have the Adiscon private repository and e.g. the testbench
changes are now only in there (it was my testbed to see which problems come
up with that approach). But I've already seen how this affects my workflow,
as I am no longer able to point people to the code as it develops (there
was a concrete case yesterday with a broken testbench test).
So far, I tried to do everything in public and not be ashamed of the
sometimes dumb errors I made and that manifest in git history. I admit that
I make "trivial" changes directly to master and every now and then
(seldom, but happend), I screwed up and got master to fail building. Not
sure, though, if that would have been any better if I create a new branch,
apply change, merge new branch, delete it - all within 5 minutes. This
sounds a little bit overboard to me.
I think that there are multiple things here.
Doing things in a separate branch has overhead.
but git is designed to do this fast and easily, so is it really that much
overhead?
Doing things privately makes it easy to change the history, but hard to
collaborate.
Doing things directly on the master branch has the potential to run into
problems if you end up with the changes for multiple features being interleved
with each other.
If I could wave a wand to make things work in the ideal way (with multiple
active developers), what I would do is to mostly copy what the development
process for git looks like.
Implement features in separate branches that are not stable (which means that
the author is able to rebase, squash, reorder, etc), but either are public, or
can be made public if needed for collaboration (which is why I don't really
think it's a good idea to shutdown the adiscon public git repository)
Then when the feature is believed to be complete, it gets merged into a
'pending' tree. This tree exists for the sole purpose of having the testbench
run against it daily. This tree is not stable, it's recreated for each day's
test run by copying the master and then merging in all the pending features that
are beleived to be ready to be tested.
If all the tests pass, the feature branch then gets merged into master (at which
point the feature branch can be deleted, but if it's a big change, may be kept
around in case fixes are needed, but if so, the merged changes can no longer be
rebased or squashed)
If all the tests do not pass, the 'pending' tree automation should bisect the
features that it's merging to identify which one is breaking it, and send a
notification (to the author of that tree at least)
In the case of highly prolific programmers, the programmer may merge their
smaller features into their own 'pending updates from X' tree and that tree can
be what's pulled, so the configuration of the 'pending' tree doesn't change.
The merging of the feature branches into the master does not have to happen
daily (after the testbench runs), it's just that they must go through at least
one testbech run before being eligiable to merge them. Tiny changes, bugfixes,
and new modules can be pulled rapidly, but for large changes, the timing of when
to pull them needs to be thought about a little more (don't pull a large change
into master the day before a release is due for example)
With the 6 week release cycle, I would look at developing/merging features
during the first few weeks of the cycle, and then towards the end of the cycle,
only bugfixes would go in, with any substantial changes just waiting (with a
copy in the pending tree) until the next cycle. The kernel is on a ~10 week
cycle, and they merge new features during the first 2-4 weeks and then no new
features get added. For Rsyslog, we don't need such a long stabilization time,
so 4 weeks of new features and 2 weeks where only bugfixes are added (but
features can continue to be developed in the separate feature branches) seems
like it's probably pleanty long. We can experiment with this over time.
Now, with Rainer doing 90+% of the work, this process seems like it's a lot of
overhead, and Rainer has done a good job of keeping the master branch where he
does a lot of his work very stable. How we want to adjust this to keep the
overhead low is a matter for debate.
As a note on CI testing, no large entitiy does a full test run against each
commit individually, this just doesn't scale. At some point the tests just take
too long to run. So they fall back to doing the tests at a larer granularity,
possibly with unit tests being run on each change (if you can define unit tests
in a way that lets you decide easily what tests are relevant to the code that
was changed)
The 'pending' tree that I describe above is done by both git and the
linux-kernel, so we should be able to get copies of their automation tools
rather than having to recreate things from scratch. So it shouldn't be as hard
to setup as it may initially sound.
As for the history being dominated by merges, this is the result of maintaining
so many different versions. As John noted when he did the documenation tree,
merging changes into the oldest supported version and then merging that into the
newer versions lets git do a LOT of work for you in how it remembers what
changes have been made and integrating the new changes into it. I think that
dropping maintinance of so many old versions is a very reasonable thing to do.
Adiscon will need to maintain some older versions for it's contracted customers,
and some distros may want to maintain older versions (or sponsor such
maintinance), but spending a lot of effort on such older versions isn't worth
very much. I think the value is low enough that the hassle of the git history
with lots of merges due to all the old versions is enough to make it
questionable.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
