Re: [PATCH] Use git pull with rebase in build script
On Sun, Jul 12, 2015 at 12:20:57PM -0700, Linus Torvalds wrote: On Sun, Jul 12, 2015 at 11:07 AM, Dirk Hohndel d...@hohndel.org wrote: It's just that I don't know how else to do it and keep the branches reasonably useful. So I guess the current model *works*, it just causes nasty problems occasionally. The best model is likely one where you have two separate branches: - you have the ugly working branch that is never ever rebased, and that people who want to track libdivecomputer as a _user_ would be (so this is what the build.sh script would pull). Because it's never rebased, git pull works fine, with or without rebase (and with --rebase might be better, because *if* somebody has their own work going on, they most definitely wouldn't want to have that work ever get mixed into that branch. - you have the cleanup branch, which ends up occasionally rebasing on top of upstream libdivecomputer, and has a clean history that you can hope that Jef will eventually take more and more of. This cleanup branch you don't necessarily work on very actively, it's more of a update when libdivecomputer has done something we really want or care about. Now, that all sounds easy (just have two branches), but the magic is that you want to keep them in sync. And *that* is fairly simple, but it's simple only if you follow a few basic rules. OK, I have tried to implement this. Let's see if I manage to keep it up without going insane. But I really like the ideas that Linus explained here and I think that this will make it easier for Jef or for any distro that would like to take our patches. So we now have (in git://git.subsurface-divelog.org/libdc) two new branches that right now are identical: Subsurface-branch (this will be the 'ugly working branch') Subsurface-clean (this will be the 'cleanup branch') As Linus explains, I plan to keep Subsurface-branch as a branch one can continue to pull from without going insane. So moving forward THIS is what developers should use. I'll update the build.sh accordingly. And Subsurface-clean will continue as a clean branch on top of libdivecomputer master. So this one will rebase every once in a while. It's NOT the branch people should use to build, it's the branch that distros can use to get our patches relative to a recent libdivecomputer master, and where Jef can easily see which patches we have on top of his master right now. (a) they start out with the exact same contents (you're a mathematician, this is going to be an inductive proof, and this is the starting point - and the trivial case is by having the exact same tree with the same history). That's what I have. Same tree, same history. (b) You need to remember this place in the ugly working tree. The *contents* are the same, but the history may not be, remember, so you might tag that location with sync-point or something like that. Done (I haven't pushed those tags... since they will move over time and since only I should have a need for them) (c) when *you* do libdivecomputer development, you probably want to work in the ugly working tree, and you just keep adding commits to it. Ignore the cleanup tree, it's not what people really *use*, it's more of a nice marker of what used to be clean development ready for merging back upstream. So work work work in this tree normally, and the two trees diverge both in history and in content. But the clean tree stays the same. (d) you decide that you want to synchronize the two trees after having done *your* work, you can now do a complex rebase git rebase -i --onto cleanup sync-point which should rebase all your work you did in (c) since your last sync-point onto the cleanup branch. NOTE! It's important that cleanup and sync-point have the exact same tree: that guarantees that the rebase will be trivial. You're applying the same patches onto the same tree state - it's just a different history. You're now moving your ugly tree work to be on top of the cleaned-up history. (e) you may choose to clean up history further now (it maybe you weren't so careful when you were working in your ugly tree, but now you want things to be tip-top, so you do some cleanup and further testing in this state, wanting to make it be something that you could send upstream to Jef. (f) in fact, it's now so nice and clean that you update the cleanup branch with the new state: git branch -f cleanup HEAD because you now have a new cleanup state. (g) but now you've rebased your ugly tree, and it might not even match the old state of your working tree because of the cleanups you did in (e), so we've really screwed up! The whole point was to make the ugly tree always be a fast-forward! And the whole point was to keep the two in sync (see step (a))! What kind of idiotic working model is this! (h) to the rescue. At this point, you just do git pull --strategy ours publicrepo
Re: [PATCH] Use git pull with rebase in build script
On Sun, Jul 12, 2015 at 11:07 AM, Dirk Hohndel d...@hohndel.org wrote: It's just that I don't know how else to do it and keep the branches reasonably useful. So I guess the current model *works*, it just causes nasty problems occasionally. The best model is likely one where you have two separate branches: - you have the ugly working branch that is never ever rebased, and that people who want to track libdivecomputer as a _user_ would be (so this is what the build.sh script would pull). Because it's never rebased, git pull works fine, with or without rebase (and with --rebase might be better, because *if* somebody has their own work going on, they most definitely wouldn't want to have that work ever get mixed into that branch. - you have the cleanup branch, which ends up occasionally rebasing on top of upstream libdivecomputer, and has a clean history that you can hope that Jef will eventually take more and more of. This cleanup branch you don't necessarily work on very actively, it's more of a update when libdivecomputer has done something we really want or care about. Now, that all sounds easy (just have two branches), but the magic is that you want to keep them in sync. And *that* is fairly simple, but it's simple only if you follow a few basic rules. (a) they start out with the exact same contents (you're a mathematician, this is going to be an inductive proof, and this is the starting point - and the trivial case is by having the exact same tree with the same history). (b) You need to remember this place in the ugly working tree. The *contents* are the same, but the history may not be, remember, so you might tag that location with sync-point or something like that. (c) when *you* do libdivecomputer development, you probably want to work in the ugly working tree, and you just keep adding commits to it. Ignore the cleanup tree, it's not what people really *use*, it's more of a nice marker of what used to be clean development ready for merging back upstream. So work work work in this tree normally, and the two trees diverge both in history and in content. But the clean tree stays the same. (d) you decide that you want to synchronize the two trees after having done *your* work, you can now do a complex rebase git rebase -i --onto cleanup sync-point which should rebase all your work you did in (c) since your last sync-point onto the cleanup branch. NOTE! It's important that cleanup and sync-point have the exact same tree: that guarantees that the rebase will be trivial. You're applying the same patches onto the same tree state - it's just a different history. You're now moving your ugly tree work to be on top of the cleaned-up history. (e) you may choose to clean up history further now (it maybe you weren't so careful when you were working in your ugly tree, but now you want things to be tip-top, so you do some cleanup and further testing in this state, wanting to make it be something that you could send upstream to Jef. (f) in fact, it's now so nice and clean that you update the cleanup branch with the new state: git branch -f cleanup HEAD because you now have a new cleanup state. (g) but now you've rebased your ugly tree, and it might not even match the old state of your working tree because of the cleanups you did in (e), so we've really screwed up! The whole point was to make the ugly tree always be a fast-forward! And the whole point was to keep the two in sync (see step (a))! What kind of idiotic working model is this! (h) to the rescue. At this point, you just do git pull --strategy ours publicrepo to say pull in all the stuff I've pushed out, but use the merge strategy of picking the exact current state. This is what makes your tree the ugly tree: you just created a nice clean branch (to use as your cleaned-up version), and then you use the state of that branch (to make sure it matches the cleanup - see point (a)), but you merged in all the ugly history, including all the old ugly merges, so that your branch is always a fast-forward. (i) Now remember this state as the new sync-point: git tag -f sync-point (or I guess you could keep sync-point as just a branch instead of a tag, that way you'll have the branch history in the reflog) And now we're back to (a). What I didn't describe above is what you should do when you actually want to update from Jef's upstream. When you do that, make sure that you synchronize like the above first, so that your clean branch and your ugly branch have the same contents. Then you just check out your clean branch, and rebase it on top of its upstream (ie Jef's branch). The clean branch you rebase, after all, and it has simple clean history so rebasing it on top of Jef's should be fairly simple too, and keeps the history clean: git checkout cleanup gti pull --rebase And after you've rebased the clean branch, you go back to your ugly branch, and since the clean
Re: [PATCH] Use git pull with rebase in build script
On Sun, Jul 12, 2015 at 03:51:39PM +0200, Henrik Brautaset Aronsen wrote: On 12 Jul 2015, at 15:45, Dirk Hohndel d...@hohndel.org wrote: Why would you need -rebase for the libgit2 repository, unless you have local changes (and I just realize that I need to patch the build script to use a later commit there, anyway, as that one still doesn't work with proxies). OK, maybe the rebase in libgit is a bit too eager. OK And since I frequently force push into the Subsurface-testing branches of libdc and marble, I'm not sure what benefit -rebase gives you there. You usually need to remove those directories and check out fresh copies for things to be in a consistent state. Rebase can actually cope with a force push automatically. It’s pretty awesome, as some would say. On several occations when the build fails, it’s in the middle of a merge in either libdc or (recently) marble. Then I just go to the project in question, abort the merge, and do a git pull —rebase. Voila, build.sh can continue. By adding this to the script I don’t have to intervene manually. Thanks for the explanation, Henrik. I read a bit through the git-rebase manpage and that has not necessarily completely enlightened me on the wisdom of this changes. Linus, this sounds nice... you obviously understand the inner workings here much better. Is this a reasonable thing to do? Or will it cause pregnant kittens? I like adding things that make it easier for our developers to stay current. /D ___ subsurface mailing list subsurface@subsurface-divelog.org http://lists.subsurface-divelog.org/cgi-bin/mailman/listinfo/subsurface
[PATCH] Use git pull with rebase in build script
From: Henrik Brautaset Aronsen subsurf...@henrik.synth.no Without rebase, the build script would stop because it couldn't complete a merge. --- scripts/build.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/build.sh b/scripts/build.sh index 7b54bea..cb9b4ad 100755 --- a/scripts/build.sh +++ b/scripts/build.sh @@ -77,7 +77,7 @@ if [ ! -d libgit2 ] ; then fi cd libgit2 # let's build with a recent enough version of master for the latest features -git pull origin master +git pull --rebase origin master if ! git checkout c11daac9de2 ; then echo Can't find the right commit in libgit2 - giving up exit 1 @@ -110,7 +110,7 @@ if [ ! -d libdivecomputer ] ; then fi fi cd libdivecomputer -git pull +git pull --rebase if ! git checkout Subsurface-testing ; then echo can't check out the Subsurface-testing branch of libdivecomputer -- giving up exit 1 @@ -134,7 +134,7 @@ if [ ! -d marble-source ] ; then fi fi cd marble-source -git pull +git pull --rebase if ! git checkout Subsurface-testing ; then echo can't check out the Subsurface-testing branch of marble -- giving up exit 1 -- 2.4.1 ___ subsurface mailing list subsurface@subsurface-divelog.org http://lists.subsurface-divelog.org/cgi-bin/mailman/listinfo/subsurface
Re: [PATCH] Use git pull with rebase in build script
On 12 Jul 2015, at 15:45, Dirk Hohndel d...@hohndel.org wrote: Why would you need -rebase for the libgit2 repository, unless you have local changes (and I just realize that I need to patch the build script to use a later commit there, anyway, as that one still doesn't work with proxies). OK, maybe the rebase in libgit is a bit too eager. And since I frequently force push into the Subsurface-testing branches of libdc and marble, I'm not sure what benefit -rebase gives you there. You usually need to remove those directories and check out fresh copies for things to be in a consistent state. Rebase can actually cope with a force push automatically. It’s pretty awesome, as some would say. On several occations when the build fails, it’s in the middle of a merge in either libdc or (recently) marble. Then I just go to the project in question, abort the merge, and do a git pull —rebase. Voila, build.sh can continue. By adding this to the script I don’t have to intervene manually. Henrik ___ subsurface mailing list subsurface@subsurface-divelog.org http://lists.subsurface-divelog.org/cgi-bin/mailman/listinfo/subsurface
Re: [PATCH] Use git pull with rebase in build script
Henrik, I'll admit that I think of the build script more as a get started, have all the pieces in place kinda thing. And then assume that people will simply update the different packages as needed (which in most cases means not very often). Why would you need -rebase for the libgit2 repository, unless you have local changes (and I just realize that I need to patch the build script to use a later commit there, anyway, as that one still doesn't work with proxies). And since I frequently force push into the Subsurface-testing branches of libdc and marble, I'm not sure what benefit -rebase gives you there. You usually need to remove those directories and check out fresh copies for things to be in a consistent state. So can you explain a bit more why you suggest this change? /D On Sun, Jul 12, 2015 at 02:42:18PM +0200, subsurf...@henrik.synth.no wrote: From: Henrik Brautaset Aronsen subsurf...@henrik.synth.no Without rebase, the build script would stop because it couldn't complete a merge. --- scripts/build.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/build.sh b/scripts/build.sh index 7b54bea..cb9b4ad 100755 --- a/scripts/build.sh +++ b/scripts/build.sh @@ -77,7 +77,7 @@ if [ ! -d libgit2 ] ; then fi cd libgit2 # let's build with a recent enough version of master for the latest features -git pull origin master +git pull --rebase origin master if ! git checkout c11daac9de2 ; then echo Can't find the right commit in libgit2 - giving up exit 1 @@ -110,7 +110,7 @@ if [ ! -d libdivecomputer ] ; then fi fi cd libdivecomputer -git pull +git pull --rebase if ! git checkout Subsurface-testing ; then echo can't check out the Subsurface-testing branch of libdivecomputer -- giving up exit 1 @@ -134,7 +134,7 @@ if [ ! -d marble-source ] ; then fi fi cd marble-source -git pull +git pull --rebase if ! git checkout Subsurface-testing ; then echo can't check out the Subsurface-testing branch of marble -- giving up exit 1 -- 2.4.1 ___ subsurface mailing list subsurface@subsurface-divelog.org http://lists.subsurface-divelog.org/cgi-bin/mailman/listinfo/subsurface ___ subsurface mailing list subsurface@subsurface-divelog.org http://lists.subsurface-divelog.org/cgi-bin/mailman/listinfo/subsurface
Re: [PATCH] Use git pull with rebase in build script
On Sun, Jul 12, 2015 at 7:05 AM, Dirk Hohndel d...@hohndel.org wrote: Thanks for the explanation, Henrik. I read a bit through the git-rebase manpage and that has not necessarily completely enlightened me on the wisdom of this changes. I suspect it ends up working better in practice, although I do think it has the chance of being really really horribly broken when you do a non-fast-forward. I dunno. The current situation isn't good either. The whole base things on a branch that does non-fast-forwards model really is entirely broken. Linus ___ subsurface mailing list subsurface@subsurface-divelog.org http://lists.subsurface-divelog.org/cgi-bin/mailman/listinfo/subsurface
