Re: Need help migrating workflow from svn to git.
Hi Josef, On 20/12/2017 12:43, Josef Wolf wrote: > >> $ git add -u >> $ git reset > > This would be added after the "git checkout -m -B master FETCH_HEAD" > command? Yes, so it would be something like this: git fetch origin master && #1 git checkout -m -B master FETCH_HEAD && #2 git add -u && #3 git reset #4 But it actually depends on what kind of default `git diff` output you prefer. In order to avoid failure on subsequent script runs, in case where conflicts still exist, you need to ensure #3 and #4 are executed before #1 and #2 are executed _again_. So you may put #3 and #4 in front of #1 and #2, too, that would work just as well, where `git diff` would now be showing "combined diff"[2] as long as the script isn`t executed again (and it would keep showing new "combined diff" from that point on). >> Yes, `git diff` won`t be the same as if conflicts were still in, but >> it might be worth it in this specific case, conflicting parts still >> easily visible between conflict markers. > > That means, the conflict is still there, but git would think this is > an ordinary modification? Yes, as by that `git add -u` you confirm all merge conflicts are resolved, and `git diff` output changes accordingly. You can read more about "diff format for merges"[1] and "combined diff format"[2] from `git-diff`[3] documentation. Here are some examples from my test repositories. Local repo introduces line "A1" (local modification, uncommitted), where remote repo introduced line "B1" (commit). Steps #1 and #2 get executed, merge conflicts shown with `git diff`, before `git add -u` and `git reset`: $ git diff diff --cc A index 5314b4f,1e2b966..000 --- a/A +++ b/A @@@ -12,5 -12,5 +12,9 @@@ 2 3 4 ++<<< FETCH_HEAD +B1 ++=== + A1 ++>>> local 5 ... and after `git add -u` and `git reset` (note line "B1" not showing as changed anymore): $ git diff diff --git a/A b/A index 5314b4f..8ea9600 100644 --- a/A +++ b/A @@ -12,5 +12,9 @@ A 2 3 4 +<<< FETCH_HEAD B1 +=== +A1 +>>> local 5 Now, without any commits yet made locally (except commit pulled from remote repo), local repo adds line "A2" where remote repo introduces line "B2" (commit). Steps #1 and #2 get executed again, merge conflicts shown with `git diff`, before `git add -u` and `git reset`: $ git diff diff --cc A index 424ae9e,4aac880..000 --- a/A +++ b/A @@@ -2,7 -2,7 +2,11 @@@ 1 2 3 ++<<< FETCH_HEAD +B2 ++=== + A2 ++>>> local 4 5 6 ... and after `git add -u` and `git reset` (note showing line "B2" as unchanged, and now showing leftover "conflicts" around "A1" here as well, where previous "combined" diff discarded it as uninteresting due to implied "--cc"[4] flag): $ git diff diff --git a/A b/A index 424ae9e..77ad8e6 100644 --- a/A +++ b/A @@ -2,7 +2,11 @@ A 1 2 3 +<<< FETCH_HEAD B2 +=== +A2 +>>> local 4 5 6 @@ -13,5 +17,9 @@ A3 2 3 4 +<<< FETCH_HEAD B1 +=== +A1 +>>> local 5 Hope that helps. As usual, best to give it some try on your own :) Regards, Buga [1] https://git-scm.com/docs/git-diff#_diff_format_for_merges [2] https://git-scm.com/docs/git-diff#_combined_diff_format [3] https://git-scm.com/docs/git-diff [4] https://git-scm.com/docs/git-diff-tree#git-diff-tree---cc
Re: Need help migrating workflow from svn to git.
On Wed, Dec 20, 2017 at 12:43:37PM +0100, Josef Wolf wrote: > Thanks to you both for your patience with me. Sorry for the late reply, my day > job was needing me ;-) > > On Fri, Dec 15, 2017 at 07:58:14PM +0100, Igor Djordjevic wrote: > > On 15/12/2017 17:33, Junio C Hamano wrote: > > > > > > $ git fetch > > > $ git checkout -m -B FETCH_HEAD > > For some reason, this seems to double the local modifications. After executing > the following commands: Umm... Please ignore this "doubling" comment. My test script was faulty :-// Sorry for the confusion! -- Josef Wolf j...@raven.inka.de
Re: Need help migrating workflow from svn to git.
On Fri, Dec 15, 2017 at 11:09:17AM -0800, Junio C Hamano wrote: > Igor Djordjevic writes: > > > Junio, what about consecutive runs, while merge conflicts are still > > unresolved? > > The impression I got was that the original running with svn does not > deal with conflicting situation anyway, so I did not think about it > at all, and I personally do not care ;-) Yeah. Conflicts are not a big deal for this project. I guess, this would no longer hold true if there are lots of developers with lots of commits. Thus, I'd still like to learn how to do it correctly. Never doing local modifications in the "live" directories would require to install test systems and simulate their environment just for doing some minor adjustments. -- Josef Wolf j...@raven.inka.de
Re: Need help migrating workflow from svn to git.
Thanks to you both for your patience with me. Sorry for the late reply, my day job was needing me ;-) On Fri, Dec 15, 2017 at 07:58:14PM +0100, Igor Djordjevic wrote: > On 15/12/2017 17:33, Junio C Hamano wrote: > > > > $ git fetch > > $ git checkout -m -B FETCH_HEAD For some reason, this seems to double the local modifications. After executing the following commands: rm -rf reposA reposB git init reposA ( cd reposA echo 1 >>1 echo 2 >>2 git add 1 2 git commit -m1 ) git clone reposA reposB ( cd reposA echo 1 >>1 git commit -a -m2 ) ( cd reposB echo 3 >>2 git fetch git checkout -m -B master FETCH_HEAD ) git-diff gives me: $ diff --git a/2 b/2 index 0cfbf08..4e8a2de 100644 --- a/2 +++ b/2 @@ -1 +1,3 @@ 2 +3 +3 With Igor's set of commands, I did not see this doubling: > git checkout -b temp && #1 > git fetch && #2 > git branch -f master origin/master && #3 > git checkout -m master && #4 > git add -u && #5 > git reset && #6 > git branch -d temp#7 > ... aaand that`s how you do it[1] without a temporary branch :) > > Junio, what about consecutive runs, while merge conflicts are still > unresolved? > > Seeing Josef having a pretty relaxed flow, and his cron job running > every 15 minutes, would adding something like: > > $ git add -u > $ git reset This would be added after the "git checkout -m -B master FETCH_HEAD" command? > ... to the mix, to "silence" actually still unresolved merge > conflicts, making next script execution possible, make sense? > > Yes, `git diff` won`t be the same as if conflicts were still in, but > it might be worth it in this specific case, conflicting parts still > easily visible between conflict markers. That means, the conflict is still there, but git would think this is an ordinary modification? -- Josef Wolf j...@raven.inka.de
Re: Need help migrating workflow from svn to git.
On 15/12/2017 20:09, Junio C Hamano wrote: > > > Junio, what about consecutive runs, while merge conflicts are still > > unresolved? > > The impression I got was that the original running with svn does not > deal with conflicting situation anyway, so I did not think about it > at all, and I personally do not care ;-) Heh, fair enough :) Though I was genuinely interested if there is a better way to accomplish it than `git add -u && git reset`, but I guess I can take that as "whatever works" ;)
Re: Need help migrating workflow from svn to git.
Igor Djordjevic writes: > Junio, what about consecutive runs, while merge conflicts are still > unresolved? The impression I got was that the original running with svn does not deal with conflicting situation anyway, so I did not think about it at all, and I personally do not care ;-)
Re: Need help migrating workflow from svn to git.
On 15/12/2017 17:33, Junio C Hamano wrote: > > $ git fetch > $ git checkout -m -B FETCH_HEAD ... aaand that`s how you do it[1] without a temporary branch :) Junio, what about consecutive runs, while merge conflicts are still unresolved? Seeing Josef having a pretty relaxed flow, and his cron job running every 15 minutes, would adding something like: $ git add -u $ git reset ... to the mix, to "silence" actually still unresolved merge conflicts, making next script execution possible, make sense? Yes, `git diff` won`t be the same as if conflicts were still in, but it might be worth it in this specific case, conflicting parts still easily visible between conflict markers. Regards, Buga [1] On 15/12/2017 19:24, Igor Djordjevic wrote: > > git checkout -b temp && #1 > git fetch && #2 > git branch -f master origin/master && #3 > git checkout -m master && #4 > git add -u && #5 > git reset && #6 > git branch -d temp#7 > > Explanation: > 1. Create temporary branch where we are, switching to it, so we can > update "master" without local modifications > 2. Fetch latest updates > 3. Update "master" to fetched "origin/master" > 4. Switch to updated "master", merging local modifications > 5. Mark any pending merge conflicts as resolved by staging them... > 6. ... and unstage them right away > 7. Delete temporary branch > > Step (4) is what merges your local modifications with remote updates > (leaving conflicts, if any), where steps (5) and (6) are not needed > for a single run, but in case you don`t resolve conflicts before next > cron job executes this script again, step (1) will now fail without > them because of (still) unresolved merge conflicts. > > So, as you seem to be pretty at ease with your flow, you might prefer > leaving those two steps (5, 6) in.
Re: Need help migrating workflow from svn to git.
Hi Josef, Thank you for your patient answers. From what you said here and in that other reply[1], it looks like you know what you`re doing, you`re aware of circumstances, and you still prefer doing it that way. So, here it goes... :) On 15/12/2017 13:47, Josef Wolf wrote: > > > I`m thinking of a workflow involving (scripted) creation of a > > temporary branch at fetched remote branch position, and using > > something like `git checkout --merge ` to merge your > > local modifications to latest changes fetched from remote (ending > > up with conflicts inside working tree, if any), > > But this would require local modifications to be committed? Nope :) Here`s a script you can test to see if it works for you, simulating `svn update` (at least how I perceived it). Feel free to adapt as you feel like it (I used local "master" branch and remote "origin/master", for example), or to speak up if any additional info is needed. git checkout -b temp && #1 git fetch && #2 git branch -f master origin/master && #3 git checkout -m master && #4 git add -u && #5 git reset && #6 git branch -d temp#7 Explanation: 1. Create temporary branch where we are, switching to it, so we can update "master" without local modifications 2. Fetch latest updates 3. Update "master" to fetched "origin/master" 4. Switch to updated "master", merging local modifications 5. Mark any pending merge conflicts as resolved by staging them... 6. ... and unstage them right away 7. Delete temporary branch Step (4) is what merges your local modifications with remote updates (leaving conflicts, if any), where steps (5) and (6) are not needed for a single run, but in case you don`t resolve conflicts before next cron job executes this script again, step (1) will now fail without them because of (still) unresolved merge conflicts. So, as you seem to be pretty at ease with your flow, you might prefer leaving those two steps (5, 6) in. This does seem ugly and hacky, but if it works for you, I don`t judge :) Please note that there might be better ways to accomplish this, I repeat, I`m not an expert, but hopefully this could do the job. Also, if I missed something, I hope someone will correct me. Regards, Buga [1] https://public-inbox.org/git/20171215130645.gd18...@raven.inka.de/
Re: Need help migrating workflow from svn to git.
Josef Wolf writes: > With git, by contrast, this won't work. Git will refuse to pull anything as > long as there are ANY local modifications. The cron job would need to > >git stash >git pull >git stash pop I'd assume that this "pull" is expected to be fast-forward, as otherwise you have no way of dealing with conflicted merges. > But this will temporarily remove my local modifications. If I happen to do > a test run at this time, the test run would NOT contain the local > modifications which I was about to test. Even worse: if I happen to save > one of the modified files while the modifications are in the stash, the > "git stash pop" will definitely cause a conflict, although nothing really > changed. > > So, how would I get this workflow with git? Is it possible to emulate the > behavior of "svn update"? You do not mind a temporary inconsistency while "svn update" runs (it starts to update a file you may have local changes, but your test may run while the update is in the middle of it). So perhaps something along the lines of this would help. Assuming : the branch at the remote you are pulling from : whatever branch you are using are in your three-command example above: $ git fetch $ git checkout -m -B FETCH_HEAD should give you pretty-much identical result as $ git stash && git pull --ff-only && git stash pop including a possible merge conflicts at 'git stash pop' stage.
Re: Need help migrating workflow from svn to git.
On Fri, Dec 15, 2017 at 02:17:40AM +0100, Igor Djordjevic wrote:
>
> This said, and without having you to change your habits too much (nor
> use Git in possibly awkward ways), I`m thinking you may actually
> benefit of using `git worktree add `[1] to create a
> temporary working tree ("working copy", as you say), alongside a
> temporary branch, where you could hack and test as much as you want,
> unaffected by cron job updating and executing the original working
> copy/branch (also not stepping on cron job`s toes yourself).
Interesting command. Did not know about it. Will remembre it for other use
cases!
But in this case, it is not of much use. See, I am doing system configuration
with those scripts. Having two working trees would cause the configuration to
be restored every time cron happens to start the scripts.
Doing the modifications right there where cron executes has the benefit that
cron uses the same modifications which I am using. This way, whenever cron
decides to execute, it is exactly the same as if I would do a "make run" on
the command line. Since all the scripts are designed to be idempotent,
everything works pretty much flawlessly.
--
Josef Wolf
j...@raven.inka.de
Re: Need help migrating workflow from svn to git.
Thanks for your input, Igor! On Thu, Dec 14, 2017 at 11:27:09PM +0100, Igor Djordjevic wrote: > Aside "update and merge" working copy while you`re hacking on it, > what happens with "execute" part? It seems really strange that you > don`t mind cron job running the same scripts which you are actively > working on, thus being in an inconsistent state, if not broken, even. In theory, you're right. In practice, problems are almost non-existent because of: 1. Scripts are independant from each other. Would there be a problem with one script, the other several hundred scripts will still continue to work. 2. Most changes to existing scripts are just minor tweaks. Not much room to wind up. 3. When new scripts/features are introduced, it is usually to some aspect that were non-existing before. Breaking something that did not work before is not a big issue. 4. Even IF cron happens to execute something half-baked, most of the time it will give a syntax error. The effect is as if the cron job would have been stopped entirely 5. When there's a major refactoring to be done where some problems are to expected, then the cron entry is disabled. So, in practice, it's pretty much a non-issue. > > With git, by contrast, this won't work. Git will refuse to pull > > anything as long as there are ANY local modifications. > > Not sure what`s happening at your end, but "ANY" part shouldn`t be > true - you can have local modifications and still execute `git pull` > successfully. > > Only if you have local modifications in files that _also_ changed on > the remote end, `git pull` aborts (fetch of the remote branch > succeeds, actually, just merge with local branch is aborted). Oh, you're right! I don't know where I catched the idea that ANY modification would stop "git pull" from working... > Now, having in mind you said conflicts are extremely rare in your > flow anyway, would this be enough for you? No. There's still the issue with "git pop", even if no modifications are coming from upstream. > > The cron job would need to > > > >git stash > >git pull > >git stash pop > > > > But this will temporarily remove my local modifications. If I happen > > to do a test run at this time, the test run would NOT contain the > > local modifications which I was about to test. Even worse: if I > > happen to save one of the modified files while the modifications are > > in the stash, the "git stash pop" will definitely cause a conflict, > > although nothing really changed. > > Is `git stash pop` causing conflicts your only concern here? How > about a situation where you save one of the modified files _after_ > `git stash pop` was successful, effectively discarding any updates > introduced by `git pull` from remote end...? It's both. With the conflict having the highest probability. > As you basically have a flow where two users (you and cron job) can > edit same files at the same time, desired outcome might be a bit > ambiguous, especially when scheduled execution of those files is > added to the mix. Yeah. That's the difference with svn: Svn won't touch the files unless there are changes coming from upstream. In contrast, git-stash WILL touch ALL locally modified files, even the files which were not touched at all upstream. So with svn, only files are at risk which are locally modified AND which have been changed upstream. With git-stash, EVERY locally modified file is at risk. > I`m thinking of a workflow involving (scripted) creation of a > temporary branch at fetched remote branch position, and using > something like `git checkout --merge ` to merge your > local modifications to latest changes fetched from remote (ending up > with conflicts inside working tree, if any), But this would require local modifications to be committed? -- Josef Wolf j...@raven.inka.de
Re: Need help migrating workflow from svn to git.
Thanks for your answer, Randall, On Thu, Dec 14, 2017 at 04:07:15PM -0500, Randall S. Becker wrote: > > You might want to consider a slight modification to your approach as > follows. > Instead of using git pull, use git fetch. > Have each system on its own branch (sys1 = my-sys1-branch, for example) so > you can track who has what. > In your scripts, consider: > git fetch > if nothing changed, done > git status > if no changes, git merge --ff master && git push origin my-sys1-branch && > done > if changes, send an email whining about the changes > your script could then (depending on your environment) git commit -a && git > merge && git push origin my-sys1-branch && done The scripts never commit. In fact, they only have read access to the remote repository. Commits are only ever done by humans manually. So it's going to be something like this: git fetch origin if [ git diff -s master origin/master ] git stash git merge -ff master git stash pop fi Unfortunately, the return code of git-diff don't seem to indicate whether they have diverged. And git-status don't seem to have an option to specify "remote is ahead of me". How would I properly check whether a merge is actually needed? -- Josef Wolf j...@raven.inka.de
Re: Need help migrating workflow from svn to git.
On 14/12/2017 23:27, Igor Djordjevic wrote:
>
> As you basically have a flow where two users (you and cron job) can
> edit same files at the same time, desired outcome might be a bit
> ambiguous, especially when scheduled execution of those files is
> added to the mix.
This said, and without having you to change your habits too much (nor
use Git in possibly awkward ways), I`m thinking you may actually
benefit of using `git worktree add `[1] to create a
temporary working tree ("working copy", as you say), alongside a
temporary branch, where you could hack and test as much as you want,
unaffected by cron job updating and executing the original working
copy/branch (also not stepping on cron job`s toes yourself).
Once you`re satisfied and you commit/merge/push your changes from
within the temporary working copy/branch, you can just delete it
(both temporary working copy and its branch), and you`re good :)
p.s. Even if you`re not familiar with Git branching and merging, it
shouldn`t take too much effort to wrap your head around it, and it`s
definitely worth it - and actually pretty easy, even more if you`re
working alone.
[1] https://git-scm.com/docs/git-worktree
Re: Need help migrating workflow from svn to git.
Hi Josef, I`m not a Git expert, and I know less of Subversion, but following your explanation, I might try to help, at least until more experienced people join. On 14/12/2017 14:09, Josef Wolf wrote: > > Every machine has a working copy of the repository in a specific > directory. A cron job (running every 15 minutes) executes "svn > update" and executes the scripts which are contained in this working > copy. > ... > Sometimes, I need to fix a problem on some host or need to implement > a new feature. For this, I go to the working copy of a host where the > change needs to be done and start haking. With svn, I don't need to > stop the cron job. "svn update" will happily merge any in-coming > changes and leave alone the files which were not modified upstream. > Conflicts with my local modifications which I am currently hacking on > are extremely rare, because the scripts are pretty much independent. > So I'm pretty much happy with this mode of operation. Aside "update and merge" working copy while you`re hacking on it, what happens with "execute" part? It seems really strange that you don`t mind cron job running the same scripts which you are actively working on, thus being in an inconsistent state, if not broken, even. > With git, by contrast, this won't work. Git will refuse to pull > anything as long as there are ANY local modifications. Not sure what`s happening at your end, but "ANY" part shouldn`t be true - you can have local modifications and still execute `git pull` successfully. Only if you have local modifications in files that _also_ changed on the remote end, `git pull` aborts (fetch of the remote branch succeeds, actually, just merge with local branch is aborted). Now, having in mind you said conflicts are extremely rare in your flow anyway, would this be enough for you? Of course, provided that issue you`re having with being unable to `git pull` with ANY local modifications, as you wrote, is resolved first. > The cron job would need to > >git stash >git pull >git stash pop > > But this will temporarily remove my local modifications. If I happen > to do a test run at this time, the test run would NOT contain the > local modifications which I was about to test. Even worse: if I > happen to save one of the modified files while the modifications are > in the stash, the "git stash pop" will definitely cause a conflict, > although nothing really changed. Is `git stash pop` causing conflicts your only concern here? How about a situation where you save one of the modified files _after_ `git stash pop` was successful, effectively discarding any updates introduced by `git pull` from remote end...? As you basically have a flow where two users (you and cron job) can edit same files at the same time, desired outcome might be a bit ambiguous, especially when scheduled execution of those files is added to the mix. > So, how would I get this workflow with git? Is it possible to emulate > the behavior of "svn update"? > > Any ideas? I`m thinking of a workflow involving (scripted) creation of a temporary branch at fetched remote branch position, and using something like `git checkout --merge ` to merge your local modifications to latest changes fetched from remote (ending up with conflicts inside working tree, if any), which would seem to simulate `svn update` as desired (if I understand it correctly), but it might be good to address some of the concerns I raised above first. Regards, Buga
RE: Need help migrating workflow from svn to git.
> On December 14, 2017 8:10 AM, Josef Wolf wrote: > Subject: Need help migrating workflow from svn to git. > > Hello folks, > > I am wondering whether/how my mode of work for a specific project > (currently based on SVN) could be transferred to git. > > I have a repository for maintaining configuration of hosts. This repository > contains several hundered scripts. Most of those scripts are don't depend on > each other. > > Every machine has a working copy of the repository in a specific directory. A > cron job (running every 15 minutes) executes "svn update" and executes the > scripts which are contained in this working copy. > > This way, I can commit changes to the main repository and all the hosts will > "download" and adopt by executing the newest revision of those scripts. > (The sripts need to be idempotent, but this is a different topic). > > NORMALLY, there are no local modifications in the working copy. Thus, > conflicts can not happen. Everything works fine. > > Sometimes, I need to fix a problem on some host or need to implement a > new feature. For this, I go to the working copy of a host where the change > needs to be done and start haking. With svn, I don't need to stop the cron > job. "svn update" will happily merge any in-coming changes and leave alone > the files which were not modified upstream. Conflicts with my local > modifications which I am currently hacking on are extremely rare, because > the scripts are pretty much independent. So I'm pretty much happy with this > mode of operation. > > With git, by contrast, this won't work. Git will refuse to pull anything as long > as there are ANY local modifications. The cron job would need to > >git stash >git pull >git stash pop > > But this will temporarily remove my local modifications. If I happen to do a > test run at this time, the test run would NOT contain the local modifications > which I was about to test. Even worse: if I happen to save one of the > modified files while the modifications are in the stash, the "git stash pop" will > definitely cause a conflict, although nothing really changed. > > So, how would I get this workflow with git? Is it possible to emulate the > behavior of "svn update"? > > Any ideas? You might want to consider a slight modification to your approach as follows. Instead of using git pull, use git fetch. Have each system on its own branch (sys1 = my-sys1-branch, for example) so you can track who has what. In your scripts, consider: git fetch if nothing changed, done git status if no changes, git merge --ff master && git push origin my-sys1-branch && done if changes, send an email whining about the changes your script could then (depending on your environment) git commit -a && git merge && git push origin my-sys1-branch && done This would allow you to track the condition of each system at your single upstream repository. Just my $0.02 Cheers. Randall\ -- Brief whoami: NonStop&UNIX developer since approximately UNIX(421664400)/NonStop(2112884442) -- In my real life, I talk too much.
