Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin  writes:
> Hi Sergey,
>
> On Tue, 27 Mar 2018, Sergey Organov wrote:
>
>> Johannes Schindelin  writes:
>> 
>> > On Mon, 12 Mar 2018, Sergey Organov wrote:
>> >
>> >> Johannes Schindelin  writes:
>> >> >
>> >> > On Wed, 7 Mar 2018, Sergey Organov wrote:
>> >> >
>> >> >> Johannes Schindelin  writes:
>> >> >> 
>> >> >> > How can your approach -- which relies *very much* on having the
>> >> >> > original parent commits -- not *require* that consistency check?
>> >> >> 
>> >> >> I don't understand what you mean, sorry. Could you please point me
>> >> >> to the *require* you talk about in the original proposal?
>> >> >
>> >> > Imagine a todo list that contains this line
>> >> >
>> >> > merge -C abcdef 123456
>> >> >
>> >> > and now the user edits it (this is an interactive rebase, after
>> >> > all), adding another merge head:
>> >> >
>> >> > merge -C abcdef 987654 123456
>> >> >
>> >> > Now your strategy would have a serious problem: to find the
>> >> > original version of 987654. If there was one.
>> >> 
>> >> We are talking about different checks then. My method has a built-in
>> >> check that Pillip's one doesn't.
>> >
>> > Since you did not bother to elaborate, I have to assume that your
>> > "built-in check" is that thing where intermediate merges can give you
>> > conflicts?
>> >
>> > If so, there is a possibility in Phillip's method for such conflicts,
>> > too: we have to perform as many 3-way merges as there are parent
>> > commits.
>> >
>> > It does make me uncomfortable to have to speculate what you meant,
>> > though.
>> 
>> It doesn't matter anymore as this check could easily be added to
>> Phillip's algorithm as well, see [1].
>> 
>> [1] https://public-inbox.org/git/87efkn6s1h@javad.com
>
> Ah, and there I was, thinking that finally you would answer my questions
> directly, instead you keep directing me elsewhere ("read that! Somewhere
> in there you will find the answer you are looking for").

Except I've copy-pasted it for /you/ from that reference in another
answer to /you/, and /you/ denied it there as being unexplained. As it
actually happens to be discussed and explained in the referenced
material, should I rather copy-paste the entire reference to fulfill
your requirements?

Here I repeat, directly again, that essential quote from that reference,
in case you forgot it:


git-rebase-first-parent --onto A' M
tree_U1'=$(git write-tree)
git merge-recursive B -- $tree_U1' B'
tree=$(git write-tree)
M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')
[ $conflicted_last_merge = "yes" ] ||
  trees-match $tree_U1' $tree || 
  stop-for-user-amendment
  
where 'git-rebase-first-parent' denotes whatever machinery is currently
being used to rebase simple non-merge commit.


> My time is a bit too valuable, and I will not continue a discussion where
> my questions are constantly deflected that way.

No deflection on my side was ever intended. The referenced discussion
actually has explanations. Maybe one whole page of reading, and it is to
be read in context, and then a few follow-ups in that discussion could
also be of interest, provided you are interested. I'm sorry should you
have no time for that.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Sergey,

On Tue, 27 Mar 2018, Sergey Organov wrote:

> Johannes Schindelin  writes:
> 
> > On Mon, 12 Mar 2018, Sergey Organov wrote:
> >
> >> Johannes Schindelin  writes:
> >> >
> >> > On Wed, 7 Mar 2018, Sergey Organov wrote:
> >> >
> >> >> Johannes Schindelin  writes:
> >> >> 
> >> >> > How can your approach -- which relies *very much* on having the
> >> >> > original parent commits -- not *require* that consistency check?
> >> >> 
> >> >> I don't understand what you mean, sorry. Could you please point me
> >> >> to the *require* you talk about in the original proposal?
> >> >
> >> > Imagine a todo list that contains this line
> >> >
> >> >  merge -C abcdef 123456
> >> >
> >> > and now the user edits it (this is an interactive rebase, after
> >> > all), adding another merge head:
> >> >
> >> >  merge -C abcdef 987654 123456
> >> >
> >> > Now your strategy would have a serious problem: to find the
> >> > original version of 987654. If there was one.
> >> 
> >> We are talking about different checks then. My method has a built-in
> >> check that Pillip's one doesn't.
> >
> > Since you did not bother to elaborate, I have to assume that your
> > "built-in check" is that thing where intermediate merges can give you
> > conflicts?
> >
> > If so, there is a possibility in Phillip's method for such conflicts,
> > too: we have to perform as many 3-way merges as there are parent
> > commits.
> >
> > It does make me uncomfortable to have to speculate what you meant,
> > though.
> 
> It doesn't matter anymore as this check could easily be added to
> Phillip's algorithm as well, see [1].
> 
> [1] https://public-inbox.org/git/87efkn6s1h@javad.com

Ah, and there I was, thinking that finally you would answer my questions
directly, instead you keep directing me elsewhere ("read that! Somewhere
in there you will find the answer you are looking for").

My time is a bit too valuable, and I will not continue a discussion where
my questions are constantly deflected that way.

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Johannes Schindelin  writes:

> Hi Sergey,
>
> On Mon, 12 Mar 2018, Sergey Organov wrote:
>
>> Johannes Schindelin  writes:
>> >
>> > On Wed, 7 Mar 2018, Sergey Organov wrote:
>> >
>> >> Johannes Schindelin  writes:
>> >> 
>> >> > How can your approach -- which relies *very much* on having the
>> >> > original parent commits -- not *require* that consistency check?
>> >> 
>> >> I don't understand what you mean, sorry. Could you please point me to
>> >> the *require* you talk about in the original proposal?
>> >
>> > Imagine a todo list that contains this line
>> >
>> >merge -C abcdef 123456
>> >
>> > and now the user edits it (this is an interactive rebase, after all),
>> > adding another merge head:
>> >
>> >merge -C abcdef 987654 123456
>> >
>> > Now your strategy would have a serious problem: to find the original
>> > version of 987654. If there was one.
>> 
>> We are talking about different checks then. My method has a built-in
>> check that Pillip's one doesn't.
>
> Since you did not bother to elaborate, I have to assume that your
> "built-in check" is that thing where intermediate merges can give you
> conflicts?
>
> If so, there is a possibility in Phillip's method for such conflicts, too:
> we have to perform as many 3-way merges as there are parent commits.
>
> It does make me uncomfortable to have to speculate what you meant,
> though.

It doesn't matter anymore as this check could easily be added to
Phillip's algorithm as well, see [1].

[1] https://public-inbox.org/git/87efkn6s1h@javad.com

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Buga,

On Fri, 16 Mar 2018, Igor Djordjevic wrote:

> [...]
>
> Yes, having more steps would mean more power/options to the user, but
> more complexity to explain to and guide him through as well, not really
> sure where the line should be drawn - for the first time, at least.

If you want to avoid having a huge discussion with me about bias, male
privilege and how unaware most men are of it, and how it excludes half the
potential usership/talented developers, and how representation matters --
and believe me, you do want to avoid this discussion -- you will want to
avoid referring to the user as a "he".

It might be true in your case. But it is also true in your case that you
are Russian. Yet you write English here, probably to avoid excluding
people from the discussion. And you should demonstrate the same courtesy
to people who do not happen to identify with the same gender as you do.

In short: every time you think of a user or a developer as "he", take a
step back and reflect how excluded *you* would feel if someone forced you
to change that to "she". That is exactly how much you exclude non-males if
you think of them as "he". Just don't.

The remedy is easy: use the gender-neutral "they". Which is, by the way,
not a modern invention as of late (in contrast to the "he" you use):
https://en.wikipedia.org/wiki/Singular_they#Older_usage

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Junio,

On Tue, 13 Mar 2018, Junio C Hamano wrote:

> Johannes Schindelin  writes:
> 
> > If so, what tooling do you have to identify quickly what to
> > cherry-pick, given merge conflicts?
> 
> It exactly is the issue I've been trying to find ideal solution for
> quite a while and not successfully.  Here is a sample thread
> 
>   https://public-inbox.org/git/xmqqeft3u0u5@gitster.mtv.corp.google.com/#t
> 
> and every message I mention "merge-fix" is relevant.
>
> [... detailed explanation of the current "band-aid" system...]

Thank you for the very thorough account. I have been struggling with this,
too, even chatting to Michael Haggerty about this at the Contributors'
Summit (where I missed you a lot). He pointed me to a blog post of his
about the very interesting concept of "obsolete markers" in Hg:

http://softwareswirl.blogspot.de/2013/05/obsolete-markers-in-mercurial.html

Granted, that concept really makes most sense for rebase, but I wonder
whether the concept could be extended to help your (and my) use case of
frequently-changing targets (for me, it is more rebase targets, for you it
is merge targets). It would probably be implemented using commit notes, to
allow for bidirectional mappings.

Ciao,
Dscho
> 
> The current band-aid system punts and indexes the merge-fix changes
> by merely a branch name.  When refs/merge-fix/X exists, what it
> means is "When branch X is merged to an integration branch, it is
> likely that the integration branch _already_ has merged an unnamed
> topic that causes semantic conflicts and requires this fix-up".
> This needs occasional manual adjustment---e.g. when the topic X
> turns out to be a lot more stable than the other topic Y that was
> causing us trouble with semantic conflicts, I may at some point
> reorder the topics and have topic X advance to 'next' before topic Y
> does.  And when that happens, when I merge X to 'next', because Y is
> not yet in 'next', I shouldn't apply refs/merge-fix/X (often, an
> attempt to cherry-pick it on top of a merge of X into 'next' would
> fail, which would be a bit of safety, but not always).  What I
> should do instead is to rename refs/merge-fix/X to refs/merge-fix/Y
> immediately before merging X to 'next', so that the cherry-pick is
> not applied.  When rebuilding 'master'->'jch'->'pu' chain, X (now in
> 'next') will be merged before Y (not in 'next') gets merged, and
> when it is Y's turn to be merged, the merge-fix I used to apply when
> merging topic X will be applied.
> 
> In the ideal world (I think I'm repeating the ideas raised in the
> thread quoted), the merge-fix database should be indexed with a pair
> of commit object names (e.g. a step in branch X that adds a new
> callsite for function frotz() and a step in branch Y that changes
> the function signature of frotz()), and teach the system to
> cherry-pick refs/merge-fix/A-B to resolve semantic conflicts, when
> both commits A and B appears in the integration branch for the first
> time.  And make sure these are kept up-to-date across rebasing of
> commits A and B.  After rebasing the topics X and Y that contained
> the commits A and B, if they became C and D, the system somehow
> needs to be able to locate the previous merge-fix that was valid for
> A-B pair when C-D pair gets merged.
> 
> 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Sergey,

On Mon, 12 Mar 2018, Sergey Organov wrote:

> Johannes Schindelin  writes:
> >
> > On Wed, 7 Mar 2018, Sergey Organov wrote:
> >
> >> Johannes Schindelin  writes:
> >> 
> >> > How can your approach -- which relies *very much* on having the
> >> > original parent commits -- not *require* that consistency check?
> >> 
> >> I don't understand what you mean, sorry. Could you please point me to
> >> the *require* you talk about in the original proposal?
> >
> > Imagine a todo list that contains this line
> >
> > merge -C abcdef 123456
> >
> > and now the user edits it (this is an interactive rebase, after all),
> > adding another merge head:
> >
> > merge -C abcdef 987654 123456
> >
> > Now your strategy would have a serious problem: to find the original
> > version of 987654. If there was one.
> 
> We are talking about different checks then. My method has a built-in
> check that Pillip's one doesn't.

Since you did not bother to elaborate, I have to assume that your
"built-in check" is that thing where intermediate merges can give you
conflicts?

If so, there is a possibility in Phillip's method for such conflicts, too:
we have to perform as many 3-way merges as there are parent commits.

It does make me uncomfortable to have to speculate what you meant, though.

> >> > What would your approach (that still has no satisfyingly trivial
> >> > explanation, in my mind)
> >> 
> >> Here is one-liner: rebase sides of the merge commit and then 3-way
> >> merge them, using original merge commit as merge base.
> >
> > But I already pointed out how that would undo a commit having been
> > dropped.
> 
> No. Not for this version. You did it for originally flawed version of
> the method, that has been already fixed by addition of "using original
> merge commit as merge base" in the above sentence, and that was the
> exact reason for [RFC v2], that in turn is explicitly stated at the
> beginning of [RFC v2].

Since there is no "interdiff" between your versions, it is unreasonably
hard to deduce this, and since you seem to be unwilling to explain... I'll
just leave it at that.

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Buga,

Igor Djordjevic  writes:

> Hi Sergey,
>
> On 16/03/2018 08:31, Sergey Organov wrote:
>> 
>> > > As I said, putting myself on the user side, I'd prefer entirely
>> > > separate first step of the algorithm, exactly as written, with
>> > > its own conflict resolution, all running entirely the same way as
>> > > it does with non-merge commits. I'm used to it and don't want to
>> > > learn something new without necessity. I.e., I'd prefer to
>> > > actually see it in two separate stages, like this:
>> > >
>> > > Rebasing mainline of the merge...
>> > > [.. possible conflicts resolution ..]
>> > > Merging in changes to side branch(es)...
>> > > [.. possible conflicts resolution ..]
>> > >
>> > > And if the second stage gives non-trivial conflicts, I'd like to
>> > > have a simple way to just do "merge -s ours " on top of
>> > > already rebased mainline of the merge and go with it. Note that
>> > > the latter is significantly different than re-merging everything
>> > > from scratch, that would be the only choice with "all-in-one"
>> > > approach, and it essentially gives me back those simple "rebase
>> > > first parent and just record other parents" semantics when
>> > > needed.
>> > 
>> > [...]
>> > 
>> > Also note that, for example, in case side branch(es) dropped some 
>> > commits (interactively or otherwise), first step alone would still 
>> > reintroduce those dropped changes, thus later possible `merge -s ours 
>> > ` would be a pretty bad "evil merge" case and a wrong thing to 
>> > do in general.
>> 
>> Except that my presumption is that the second step has been run already
>> and has stopped due to conflicts, so I see the conflicting result of
>> dropping those commits on side branch(es), check the previous state of
>> the right side of the conflicting merge, and decide those state, being
>> the result of the fist step after possibly demanding conflicts
>> resolution, is fine after all. Thus I just re-merge -x ours the
>> branch(es), instead of re-merging everythig from scratch only to finally
>> get back to the same result, be it evil or not, the hard way.
>
> Might be my comment missed the point here, it should have been more 
> about what you said regarding "first step having its own conflict 
> resolution" - in case of dropped commits on side branch(es), you would 
> be trying to resolve conflicts using one tree that doesn`t/shouldn`t 
> even exist anymore (rebased merge commit first parent changes), which 
> might be pretty confusing, only to find the "second stage" later 
> removing changes that you might have actually picked as "first stage" 
> conflict resolution, making it all even worse.
>
> Only once "huge merge" is done completely (meaning all steps involved 
> in merge commit rebasing), user can have a more realistic overview of 
> (possibly nested, even) conflicts to resolve (and knowing his resolution 
> will actually stick).
>
> Regarding `merge -s ours ` you mention, as you say it would 
> happen only after "huge merge" is complete (with possible conflicts), 
> I guess it`s unrelated to having "merge commit rebasing" happen in 
> one go ("huge merge"), or iteratively, in stages (from user`s 
> perspective, unrelated to underlying implementation)...?
>
> Thus I`m questioning use-case for step-by-step merge commit rebasing 
> where each stage has its own conflict resolution, in the face of it 
> possibly being more confusing than helpful.
>
> Otherwise, I see the point in what you would like to accomplish with 
> that `merge -s ours ` (not from scratch), but I`m not sure 
> what would be the most sane way to allow it, and if it would be worth 
> it in the first place, seeming to be a pretty exotic use case.

I find your arguments sound, except that I somehow feel that "exotic use
case" will solve 90% of conflicting merge rebases, as merges are about
the mainline in the first place. It's likely nothing more than personal
feeling though, so I'd simply agree to disagree at this point, at least
until some actual experience is gained.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 16/03/2018 08:31, Sergey Organov wrote:
> 
> > > As I said, putting myself on the user side, I'd prefer entirely
> > > separate first step of the algorithm, exactly as written, with
> > > its own conflict resolution, all running entirely the same way as
> > > it does with non-merge commits. I'm used to it and don't want to
> > > learn something new without necessity. I.e., I'd prefer to
> > > actually see it in two separate stages, like this:
> > >
> > > Rebasing mainline of the merge...
> > > [.. possible conflicts resolution ..]
> > > Merging in changes to side branch(es)...
> > > [.. possible conflicts resolution ..]
> > >
> > > And if the second stage gives non-trivial conflicts, I'd like to
> > > have a simple way to just do "merge -s ours " on top of
> > > already rebased mainline of the merge and go with it. Note that
> > > the latter is significantly different than re-merging everything
> > > from scratch, that would be the only choice with "all-in-one"
> > > approach, and it essentially gives me back those simple "rebase
> > > first parent and just record other parents" semantics when
> > > needed.
> > 
> > [...]
> > 
> > Also note that, for example, in case side branch(es) dropped some 
> > commits (interactively or otherwise), first step alone would still 
> > reintroduce those dropped changes, thus later possible `merge -s ours 
> > ` would be a pretty bad "evil merge" case and a wrong thing to 
> > do in general.
> 
> Except that my presumption is that the second step has been run already
> and has stopped due to conflicts, so I see the conflicting result of
> dropping those commits on side branch(es), check the previous state of
> the right side of the conflicting merge, and decide those state, being
> the result of the fist step after possibly demanding conflicts
> resolution, is fine after all. Thus I just re-merge -x ours the
> branch(es), instead of re-merging everythig from scratch only to finally
> get back to the same result, be it evil or not, the hard way.

Might be my comment missed the point here, it should have been more 
about what you said regarding "first step having its own conflict 
resolution" - in case of dropped commits on side branch(es), you would 
be trying to resolve conflicts using one tree that doesn`t/shouldn`t 
even exist anymore (rebased merge commit first parent changes), which 
might be pretty confusing, only to find the "second stage" later 
removing changes that you might have actually picked as "first stage" 
conflict resolution, making it all even worse.

Only once "huge merge" is done completely (meaning all steps involved 
in merge commit rebasing), user can have a more realistic overview of 
(possibly nested, even) conflicts to resolve (and knowing his resolution 
will actually stick).

Regarding `merge -s ours ` you mention, as you say it would 
happen only after "huge merge" is complete (with possible conflicts), 
I guess it`s unrelated to having "merge commit rebasing" happen in 
one go ("huge merge"), or iteratively, in stages (from user`s 
perspective, unrelated to underlying implementation)...?

Thus I`m questioning use-case for step-by-step merge commit rebasing 
where each stage has its own conflict resolution, in the face of it 
possibly being more confusing than helpful.

Otherwise, I see the point in what you would like to accomplish with 
that `merge -s ours ` (not from scratch), but I`m not sure 
what would be the most sane way to allow it, and if it would be worth 
it in the first place, seeming to be a pretty exotic use case.

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Buga,

Igor Djordjevic  writes:

> Hi Sergey,

[...]

>> As I said, putting myself on the user side, I'd prefer entirely separate
>> first step of the algorithm, exactly as written, with its own conflict
>> resolution, all running entirely the same way as it does with non-merge
>> commits. I'm used to it and don't want to learn something new without
>> necessity. I.e., I'd prefer to actually see it in two separate stages,
>> like this:
>> 
>> Rebasing mainline of the merge...
>> [.. possible conflicts resolution ..]
>> Merging in changes to side branch(es)...
>> [.. possible conflicts resolution ..]
>> 
>> And if the second stage gives non-trivial conflicts, I'd like to have a
>> simple way to just do "merge -s ours " on top of already rebased
>> mainline of the merge and go with it. Note that the latter is
>> significantly different than re-merging everything from scratch, that
>> would be the only choice with "all-in-one" approach, and it essentially
>> gives me back those simple "rebase first parent and just record other
>> parents" semantics when needed.
>
> I`m undecided here, and while I do see a point in what you`re saying, 
> this being new to general public I dont`t think you being accustomed 
> to it is a very strong argument :)

Sure. It's mostly that having already familiar step separate seems to be
a good idea, as well as resulting isolation of the new stuff, where I
readily agree not to granulate it further. As if the latter actually
makes any difference... Octopus merges? I mean, really?

> Yes, having more steps would mean more power/options to the user, but 
> more complexity to explain to and guide him through as well, not really 
> sure where the line should be drawn - for the first time, at least.

A good thing is that while it runs smoothly it still runs smoothly both
ways.

> Also note that, for example, in case side branch(es) dropped some 
> commits (interactively or otherwise), first step alone would still 
> reintroduce those dropped changes, thus later possible `merge -s ours 
> ` would be a pretty bad "evil merge" case and a wrong thing to 
> do in general.

Except that my presumption is that the second step has been run already
and has stopped due to conflicts, so I see the conflicting result of
dropping those commits on side branch(es), check the previous state of
the right side of the conflicting merge, and decide those state, being
the result of the fist step after possibly demanding conflicts
resolution, is fine after all. Thus I just re-merge -x ours the
branch(es), instead of re-merging everythig from scratch only to finally
get back to the same result, be it evil or not, the hard way.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 15/03/2018 08:52, Sergey Organov wrote:
> 
> > > 2. The U1' == U2' consistency check in RFC that I still think is worth
> > > to be implemented.
> >
> > At the moment, I think we`d appreciate test cases where it actually 
> > proves useful, as the general consensus seems to be leaning towards 
> > it possibly being annoying (over-paranoid).
> 
> As we now have a simple way to actually check it even in this algorithm,
> I'd suggest command-line option to either relax or enforce the check,
> whatever the default is. For the default, I'd still opt for safety, as
> without it we will gather little experience with this new matter.
> 
> Honestly, without this check available, I'd likely vote for at least an
> option for stopping on every rebased merge, on the ground that if
> rebasing a non-merge could be a trouble, rebasing a merge is at least
> double-trouble, and it's not that frequent anyway. So the check we
> discuss is actually a way to make all the process much less paranoid,
> not more.
> 
> By the way, nobody yet commented about "rerere" behavior that basically
> stops rebasing every time it fires. Do you consider it over-paranoid?

I wouldn`t really know, my workflows are usually/still rather simple, I 
don`t think I`ve ever used it on purpose, and I don`t really remember 
I`ve triggered it by accident, not having it stop for amendment, at least.

You did say you find it annoying yourself, though ;) But also accepting 
it as something that probably has a good reason, though (thus not 
considering it over-paranoid, even if annoying).

> As for test cases, I have none myself, but "-s ours" merge may be an
> example of an actual trouble.
> 
> If we don't treat it specially, then changes to side branch will be
> silently propagated over the merge, that's obviously not what is needed,
> provided user keeps his intention to leave the merge "-s ours".
> 
> If we do treat it specially, it could be the case that the merge in
> question only looks like "-s ours" by pure accident, and thus changes to
> the side branch should be propagated.
> 
> I don't see how we can safely proceed without stop for user assistance.
> Had we already achieved some consensus on this issue?

I don`t know, from what Johannes said in the past, I got an 
impression that this is to be expected ("by design"), and not worth 
bothering to stop for. And he is one of the heaviest users of (merge) 
rebasing I know.

Personally, I still feel it would make sense to stop in case like 
this, indeed, but it`s just my humble (and not necessarily much 
educated) opinion.

> > > Finally, here is a sketch of the implementation that I'd suggest to
> > > use:
> > >
> > > git-rebase-first-parent --onto A' M
> > > tree_U1'=$(git write-tree)
> > > git merge-recursive B -- $tree_U1' B'
> > > tree=$(git write-tree)
> > > M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')
> > > [ $conflicted_last_merge = "yes" ] ||
> > >   trees-match $tree_U1' $tree || 
> > >   stop-for-user-amendment
> >
> > Yes, in case where we would want the "no-op merge" check (equivalent 
> > to U1' == U2' with original approach), this aligns with something I 
> > would expect.
> >
> > Note that all the "rebase merge commit" steps leading to the check 
> > will/should probably be observed as a single one from user`s perspective 
> > (in worst case ending with nested conflicts we discussed), thus 
> > `$conflicted_last_merge` is not related to `merge-recursive` step(s) 
> > only, but `rebase-first-parent`, too (just in case this isn`t implied).
> >
> > Might be easier to reason about simply as `[ $conflicts = "yes" ] || `
> 
> No. For this check it's essential to ensure that no tweaking of the
> content has been performed under the hood after the user has resolved
> conflicts, i.e., after he has been involved last time.
> 
> If all this is done in one "huge merge" step from user point of view,
> then the check belongs to this merge, as this is the last (and the only)
> one. If it's done in steps (and I vote for it), only the last merge
> status is essential for the check, preceding merges don't matter.

"Huge merge" step (from user point of view) is exactly how I perceived 
Johannes` opinion on it, describing it`s already part of Git user 
experience (with possible nested conflicts), while otherwise possibly 
hard to explain where we are precisely at in the moment of stopping for 
(intermediate) conflict resolution.

Thus only `$conflicts`, meaning anything in the "huge merge", as no user 
action/tweaking/involvement can happen until the "huge merge" is done.

> As I said, putting myself on the user side, I'd prefer entirely separate
> first step of the algorithm, exactly as written, with its own conflict
> resolution, all running entirely the same way as it does with non-merge
> commits. I'm used to it and don't want to learn something new without
> necessity. I.e., I'd prefer to actually see it in two separate stages,
> like this:
> 
> Rebasing mainline of the 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Buga,

Igor Djordjevic  writes:

> Hi Sergey,
>
> On 14/03/2018 08:21, Sergey Organov wrote:
>> 
>> There are still 2 issues about the implementation that need to be
>> discussed though:
>> 
>> 1. Still inverted order of the second merge compared to RFC.
>> 
>> It'd be simple to "fix" again, except I'm not sure it'd be better, and
>> as there is no existing experiences with this step to follow, it
>> probably should be left as in the original, where it means "merge the
>> changes made in B' (w.r.t B) into our intermediate version of the
>> resulting merge".
>> 
>> The original Phillip's version seems to better fit the asymmetry between
>> mainline and side-branch handling.
>> 
>> The actual difference will be only in the order of ours vs theirs in
>> conflicts though, and thus it's not that critical.
>
> Shouldn`t this be easy to solve just by changing the order of  
> and , on passing to `git merge-recursive`, if needed? (or 
> that`s what you meant by "simple to fix"?)

Yes, that's exactly what I meant, except it looks cleaner as is, so I
don't think the exchange is called for.

>> 2. The U1' == U2' consistency check in RFC that I still think is worth
>> to be implemented.
>
> At the moment, I think we`d appreciate test cases where it actually 
> proves useful, as the general consensus seems to be leaning towards 
> it possibly being annoying (over-paranoid).

As we now have a simple way to actually check it even in this algorithm,
I'd suggest command-line option to either relax or enforce the check,
whatever the default is. For the default, I'd still opt for safety, as
without it we will gather little experience with this new matter.

Honestly, without this check available, I'd likely vote for at least an
option for stopping on every rebased merge, on the ground that if
rebasing a non-merge could be a trouble, rebasing a merge is at least
double-trouble, and it's not that frequent anyway. So the check we
discuss is actually a way to make all the process much less paranoid,
not more.

By the way, nobody yet commented about "rerere" behavior that basically
stops rebasing every time it fires. Do you consider it over-paranoid?

As for test cases, I have none myself, but "-s ours" merge may be an
example of an actual trouble.

If we don't treat it specially, then changes to side branch will be
silently propagated over the merge, that's obviously not what is needed,
provided user keeps his intention to leave the merge "-s ours".

If we do treat it specially, it could be the case that the merge in
question only looks like "-s ours" by pure accident, and thus changes to
the side branch should be propagated.

I don't see how we can safely proceed without stop for user assistance.
Had we already achieved some consensus on this issue?

>
>> In application to the method being discussed, we only need the check if
>> the final merge went without conflicts, so the user was not already
>> involved, and the check itself is then pretty simple:
>> 
>>  "proceed without stop only if $tree = $tree_U1'"
>> 
>> Its equivalence to the U1' == U2' test in the RFC follows from the fact
>> that if M' is non-conflicting merge of U1' and U2', then M' == U1' if
>> and only if U2' == U1'.
>
> Nicely spot! I`m glad there`s still (kind of) former U1' == U2' check 
> in this approach, too, in case it proves useful :)
>
>> Finally, here is a sketch of the implementation that I'd suggest to
>> use:
>> 
>> git-rebase-first-parent --onto A' M
>> tree_U1'=$(git write-tree)
>> git merge-recursive B -- $tree_U1' B'
>> tree=$(git write-tree)
>> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')
>> [ $conflicted_last_merge = "yes" ] ||
>>   trees-match $tree_U1' $tree || 
>>   stop-for-user-amendment
>
> Yes, in case where we would want the "no-op merge" check (equivalent 
> to U1' == U2' with original approach), this aligns with something I 
> would expect.
>
> Note that all the "rebase merge commit" steps leading to the check 
> will/should probably be observed as a single one from user`s perspective 
> (in worst case ending with nested conflicts we discussed), thus 
> `$conflicted_last_merge` is not related to `merge-recursive` step(s) 
> only, but `rebase-first-parent`, too (just in case this isn`t implied).
>
> Might be easier to reason about simply as `[ $conflicts = "yes" ] || `

No. For this check it's essential to ensure that no tweaking of the
content has been performed under the hood after the user has resolved
conflicts, i.e., after he has been involved last time.

If all this is done in one "huge merge" step from user point of view,
then the check belongs to this merge, as this is the last (and the only)
one. If it's done in steps (and I vote for it), only the last merge
status is essential for the check, preceding merges don't matter.

As I said, putting myself on the user side, I'd prefer entirely separate
first step of the algorithm, exactly as written, with its own conflict
resolution, all 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 14/03/2018 08:21, Sergey Organov wrote:
> 
> There are still 2 issues about the implementation that need to be
> discussed though:
> 
> 1. Still inverted order of the second merge compared to RFC.
> 
> It'd be simple to "fix" again, except I'm not sure it'd be better, and
> as there is no existing experiences with this step to follow, it
> probably should be left as in the original, where it means "merge the
> changes made in B' (w.r.t B) into our intermediate version of the
> resulting merge".
> 
> The original Phillip's version seems to better fit the asymmetry between
> mainline and side-branch handling.
> 
> The actual difference will be only in the order of ours vs theirs in
> conflicts though, and thus it's not that critical.

Shouldn`t this be easy to solve just by changing the order of  
and , on passing to `git merge-recursive`, if needed? (or 
that`s what you meant by "simple to fix"?)

> 2. The U1' == U2' consistency check in RFC that I still think is worth
> to be implemented.

At the moment, I think we`d appreciate test cases where it actually 
proves useful, as the general consensus seems to be leaning towards 
it possibly being annoying (over-paranoid).

> In application to the method being discussed, we only need the check if
> the final merge went without conflicts, so the user was not already
> involved, and the check itself is then pretty simple:
> 
>  "proceed without stop only if $tree = $tree_U1'"
> 
> Its equivalence to the U1' == U2' test in the RFC follows from the fact
> that if M' is non-conflicting merge of U1' and U2', then M' == U1' if
> and only if U2' == U1'.

Nicely spot! I`m glad there`s still (kind of) former U1' == U2' check 
in this approach, too, in case it proves useful :)

> Finally, here is a sketch of the implementation that I'd suggest to
> use:
> 
> git-rebase-first-parent --onto A' M
> tree_U1'=$(git write-tree)
> git merge-recursive B -- $tree_U1' B'
> tree=$(git write-tree)
> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')
> [ $conflicted_last_merge = "yes" ] ||
>   trees-match $tree_U1' $tree || 
>   stop-for-user-amendment

Yes, in case where we would want the "no-op merge" check (equivalent 
to U1' == U2' with original approach), this aligns with something I 
would expect.

Note that all the "rebase merge commit" steps leading to the check 
will/should probably be observed as a single one from user`s perspective 
(in worst case ending with nested conflicts we discussed), thus 
`$conflicted_last_merge` is not related to `merge-recursive` step(s) 
only, but `rebase-first-parent`, too (just in case this isn`t implied).

Might be easier to reason about simply as `[ $conflicts = "yes" ] || `

> where 'git-rebase-first-parent' denotes whatever machinery is currently
> being used to rebase simple non-merge commit. Handy approximation of
> which for stand-alone scripting is:
> 
> git checkout --detach A' && git cherry-pick -m 1 M
> 
> [As an interesting note, observe how, after all, that original Johannes
> Sixt's idea of rebasing of merge commit by cherry-picking its first
> parent is back there.]

Heh ;) It`s always a bit enlightening when people start from different 
positions, opposing ones, even (or so it may seem, at least), but 
eventually end up in the same place, through means of open (minded) 
discussion.

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Buga,

Igor Djordjevic  writes:

> Hi Sergey,
>
> On 13/03/2018 17:10, Sergey Organov wrote:
>> 
>> > Hi Sergey, I've been following this discussion from the sidelines,
>> > though I haven't had time to study all the posts in this thread in
>> > detail. I wonder if it would be helpful to think of rebasing a merge as
>> > merging the changes in the parents due to the rebase back into the
>> > original merge. So for a merge M with parents A B C that are rebased to
>> > A' B' C' the rebased merge M' would be constructed by (ignoring shell
>> > quoting issues)
>> >
>> > git checkout --detach M
>> > git merge-recursive A -- M A'
>> > tree=$(git write-tree)
>> > git merge-recursive B -- $tree B'
>> > tree=$(git write-tree)
>> > git merge-recursive C -- $tree C'
>> > tree=$(git write-tree)
>> > M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
>> 
>> I wonder if it's OK to exchange the order of heads in the first merge
>> (also dropped C for brevity):
>
> It should be, being "left" or "right" hand side ("theirs" or "ours") 
> of the three-way merge shouldn`t matter, they`re still both equally 
> compared to the merge-base.
>
>> git checkout --detach A'
>> git merge-recursive A -- A' M
>> tree=$(git write-tree)
>> git merge-recursive B -- $tree B'
>> tree=$(git write-tree)
>> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')
>> 
>> If so, don't the first 2 lines now read: "rebase (first parent of) M on
>> top of A'"?
>
> Hmm, lol, yes...? :) So basically, this:
>
> (1)   git checkout --detach M
>   git merge-recursive A -- M A'
>   tree=$(git write-tree)
>   ...
>
> ... is equivalent to this:
>
> (2)   git checkout --detach A'
>   git merge-recursive A -- A' M
>   tree=$(git write-tree)
>   ...
>
> ..., being equivalent to this:
>
> (3)   git checkout --detach A'
>   git cherry-pick -m 1 M
>   tree=$(git write-tree)
>   ...
>
> ..., where in all three cases that `$tree` is equivalent to U1' we 
> discussed about so much already :)

Exactly, and thanks for noticing that it's actually U1', that happens to
soon become rather handy, see below.

> I tested it like this as well, slightly modifying previously sent out 
> script (like this one[1]), and it still seems to be working ;) Nice!

Very nice of you, thanks!

Yet another outcome of this transformation is that the fist step is now
free to (and probably should) utilize all the options (-s, -X, etc.)
that usual rebase has:

git-rebase-first-parent --onto A' M
tree=$(git write-tree)

where 'git-rebase-first-parent' is whatever machinery is currently being
used to rebase simple non-merge commit.

[

Moreover, Phillip's method could further be transformed to what is in
RFC, not that I think it should, see below. Just for the sake of
completeness though, here is the essential missing transformation that
makes Phillip's method symmetric, after which it becomes true special
case of the RFC with particular rebase-first-parent implementation:

git checkout --detach A'
git merge-recursive A -- A' M
tree_U1'=$(git write-tree)
git checkout --detach B'
git merge-recursive B -- B' M
tree_U2'=$(git write-tree)
git merge-recursive M -- $tree_U1' $tree_U2'
tree=$(git write-tree)
M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')

]

>
>> If so, then it could be implemented so that it reduces back to regular
>> rebase of non-merges when applied to a single-parent commit, similar to
>> the method in the RFC, striking out one of advantages of the RFC.
>
> I guess so, but I think it now boils down only to what one finds 
> easier to reason about even more.

I actually think there is more to it. It's incremental asymmetric nature
of the Phillip's approach that I now find rather appealing and worth to
be used in practice.

While the RFC approach, being entirely symmetric, is nice from the POV
of theory and reasoning, yet simple to implement, the actual user
interface of Git is inherently asymmetric with respect to merges (one
merges side-branch(es) to mainline), so asymmetric approach of the
Phillip's method should give smoother user experience, even if only
because of 1 less merge.

There are still 2 issues about the implementation that need to be
discussed though:

1. Still inverted order of the second merge compared to RFC.

It'd be simple to "fix" again, except I'm not sure it'd be better, and
as there is no existing experiences with this step to follow, it
probably should be left as in the original, where it means "merge the
changes made in B' (w.r.t B) into our intermediate version of the
resulting merge".

The original Phillip's version seems to better fit the asymmetry between
mainline and side-branch handling.

The actual difference will be only in the order of ours vs theirs in
conflicts though, and thus it's not that critical.

2. The U1' == U2' consistency check in RFC that I still think is worth
to be implemented.

In application to the method being discussed, we only need the 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 13/03/2018 17:10, Sergey Organov wrote:
> 
> > Hi Sergey, I've been following this discussion from the sidelines,
> > though I haven't had time to study all the posts in this thread in
> > detail. I wonder if it would be helpful to think of rebasing a merge as
> > merging the changes in the parents due to the rebase back into the
> > original merge. So for a merge M with parents A B C that are rebased to
> > A' B' C' the rebased merge M' would be constructed by (ignoring shell
> > quoting issues)
> >
> > git checkout --detach M
> > git merge-recursive A -- M A'
> > tree=$(git write-tree)
> > git merge-recursive B -- $tree B'
> > tree=$(git write-tree)
> > git merge-recursive C -- $tree C'
> > tree=$(git write-tree)
> > M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
> 
> I wonder if it's OK to exchange the order of heads in the first merge
> (also dropped C for brevity):

It should be, being "left" or "right" hand side ("theirs" or "ours") 
of the three-way merge shouldn`t matter, they`re still both equally 
compared to the merge-base.

> git checkout --detach A'
> git merge-recursive A -- A' M
> tree=$(git write-tree)
> git merge-recursive B -- $tree B'
> tree=$(git write-tree)
> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')
> 
> If so, don't the first 2 lines now read: "rebase (first parent of) M on
> top of A'"?

Hmm, lol, yes...? :) So basically, this:

(1) git checkout --detach M
git merge-recursive A -- M A'
tree=$(git write-tree)
...

... is equivalent to this:

(2) git checkout --detach A'
git merge-recursive A -- A' M
tree=$(git write-tree)
...

..., being equivalent to this:

(3) git checkout --detach A'
git cherry-pick -m 1 M
tree=$(git write-tree)
...

..., where in all three cases that `$tree` is equivalent to U1' we 
discussed about so much already :)

I tested it like this as well, slightly modifying previously sent out 
script (like this one[1]), and it still seems to be working ;) Nice!

> If so, then it could be implemented so that it reduces back to regular
> rebase of non-merges when applied to a single-parent commit, similar to
> the method in the RFC, striking out one of advantages of the RFC.

I guess so, but I think it now boils down only to what one finds 
easier to reason about even more.

I`m just glad we got to U1' from this perspective as well, hopefully 
adding even more faith in the overall concept, being beaten from both 
ends and dropping out to be the same (minus minor implementation details).

Regards, Buga

[1] https://public-inbox.org/git/872944c4-ca97-9f55-a424-86d1e3299...@gmail.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Junio C Hamano]

Johannes Schindelin  writes:

> So essentially, what your cherry-pick'able commits are is a way to store
> what rerere would have stored (i.e. the set of merge conflicts together
> with their resolution)?

If rerere would have stored, I wouldn't have separate band-aid
system on top.  These fix-up commits are usually on parts that do
not get involved in textual conflicts; "rerere" which relies on
having textual conflicts (the "shape" of the text in the conflicted
region is what lets "rerere" index into its database to find the
recorded resolution) wouldn't have stored them and that is
fundamental.

> If so, what tooling do you have to identify quickly what to cherry-pick,
> given merge conflicts?

It exactly is the issue I've been trying to find ideal solution for
quite a while and not successfully.  Here is a sample thread

  https://public-inbox.org/git/xmqqeft3u0u5@gitster.mtv.corp.google.com/#t

and every message I mention "merge-fix" is relevant.

The current band-aid system punts and indexes the merge-fix changes
by merely a branch name.  When refs/merge-fix/X exists, what it
means is "When branch X is merged to an integration branch, it is
likely that the integration branch _already_ has merged an unnamed
topic that causes semantic conflicts and requires this fix-up".
This needs occasional manual adjustment---e.g. when the topic X
turns out to be a lot more stable than the other topic Y that was
causing us trouble with semantic conflicts, I may at some point
reorder the topics and have topic X advance to 'next' before topic Y
does.  And when that happens, when I merge X to 'next', because Y is
not yet in 'next', I shouldn't apply refs/merge-fix/X (often, an
attempt to cherry-pick it on top of a merge of X into 'next' would
fail, which would be a bit of safety, but not always).  What I
should do instead is to rename refs/merge-fix/X to refs/merge-fix/Y
immediately before merging X to 'next', so that the cherry-pick is
not applied.  When rebuilding 'master'->'jch'->'pu' chain, X (now in
'next') will be merged before Y (not in 'next') gets merged, and
when it is Y's turn to be merged, the merge-fix I used to apply when
merging topic X will be applied.

In the ideal world (I think I'm repeating the ideas raised in the
thread quoted), the merge-fix database should be indexed with a pair
of commit object names (e.g. a step in branch X that adds a new
callsite for function frotz() and a step in branch Y that changes
the function signature of frotz()), and teach the system to
cherry-pick refs/merge-fix/A-B to resolve semantic conflicts, when
both commits A and B appears in the integration branch for the first
time.  And make sure these are kept up-to-date across rebasing of
commits A and B.  After rebasing the topics X and Y that contained
the commits A and B, if they became C and D, the system somehow
needs to be able to locate the previous merge-fix that was valid for
A-B pair when C-D pair gets merged.

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Phillip,

Phillip Wood  writes:

[...]

> Hi Sergey, I've been following this discussion from the sidelines,
> though I haven't had time to study all the posts in this thread in
> detail. I wonder if it would be helpful to think of rebasing a merge as
> merging the changes in the parents due to the rebase back into the
> original merge. So for a merge M with parents A B C that are rebased to
> A' B' C' the rebased merge M' would be constructed by (ignoring shell
> quoting issues)
>
> git checkout --detach M
> git merge-recursive A -- M A'
> tree=$(git write-tree)
> git merge-recursive B -- $tree B'
> tree=$(git write-tree)
> git merge-recursive C -- $tree C'
> tree=$(git write-tree)
> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')

I wonder if it's OK to exchange the order of heads in the first merge
(also dropped C for brevity):

git checkout --detach A'
git merge-recursive A -- A' M
tree=$(git write-tree)
git merge-recursive B -- $tree B'
tree=$(git write-tree)
M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')

If so, don't the first 2 lines now read: "rebase (first parent of) M on
top of A'"?

If so, then it could be implemented so that it reduces back to regular
rebase of non-merges when applied to a single-parent commit, similar to
the method in the RFC, striking out one of advantages of the RFC.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin  writes:
> Hi Sergey,
>
> On Wed, 7 Mar 2018, Sergey Organov wrote:
>
>> Johannes Schindelin  writes:
>> 
>> > How can your approach -- which relies *very much* on having the
>> > original parent commits -- not *require* that consistency check?
>> 
>> I don't understand what you mean, sorry. Could you please point me to
>> the *require* you talk about in the original proposal?
>
> Imagine a todo list that contains this line
>
>   merge -C abcdef 123456
>
> and now the user edits it (this is an interactive rebase, after all),
> adding another merge head:
>
>   merge -C abcdef 987654 123456
>
> Now your strategy would have a serious problem: to find the original
> version of 987654. If there was one.

We are talking about different checks then. My method has a built-in
check that Pillip's one doesn't. All the external checks, if any, will
have to be the same.

>
>> > What would your approach (that still has no satisfyingly trivial
>> > explanation, in my mind)
>> 
>> Here is one-liner: rebase sides of the merge commit and then 3-way
>> merge them, using original merge commit as merge base.
>
> But I already pointed out how that would undo a commit having been
> dropped.

No. Not for this version. You did it for originally flawed version of
the method, that has been already fixed by addition of "using original
merge commit as merge base" in the above sentence, and that was the
exact reason for [RFC v2], that in turn is explicitly stated at the
beginning of [RFC v2].

>
>> > do if somebody edited a `merge` command and let it merge a completely
>> > unrelated commit?
>> 
>> Don't see a problem, sorry. The method should still work, provided you have
>> original merge commit and two new parents for the new merge.
>
> That is assuming a lot. That is exactly what this consistency check is
> for, that I mentioned earlier, and which you listed as a downside of
> Phillip's strategy (forgetting that your strategy has the same downside,
> so...).

Again, we are talking about different checks. My method has a built-in
check that Pillip's doesn't. All the external checks, if any, will have
to be the same.

> But I guess that you are still talking about the non-interactive version
> of the rebase, and missed that our conversation proceeded to the point
> where we want that same strategy to work *also* in the interactive version
> (and not have a completely different functionality depending whether you
> use --interactive or not)?

For me, non-interactive is an interactive one with unmodified todo list.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Junio,

On Thu, 8 Mar 2018, Junio C Hamano wrote:

> Johannes Schindelin  writes:
> 
> >> Non-textual semantic conflicts are made (in the best case just once)
> >> as a separate commit on top of mechanical auto-merge whose focus is
> >> predictability (rather than cleverness) done by Git, and then that
> >> separate commit is kept outside the history.  When replaying these
> >> merges to rebuild the 'pu' branch, after resetting the tip to
> >> 'master', each topic is merged mechanically, and if such a fix-up
> >> commit is present, "cherry-pick --no-commit" applies it and then
> >> "commit --amend --no-edit" to adjust the merge.  I find it quite
> >> valuable to have a separate record of what "evil" non-mechanical
> >> adjustment was done, which I know won't be lost in the noise when
> >> these merges need to be redone daily or more often.
> >
> > So essentially, you have something that `git rerere` would have learned,
> > but as a commit?
> 
> You probably wouldn't be asking that if you read what you cut out
> when you quoted above ;-) 

Sorry to be so stupid, and no, the part I cut did not clarify it for me,
hence my question.

> There are a collection of cherry-pickable commits in hierarchy under
> refs/merge-fix.  They are indexed by the branch that will cause
> semantic conflicts that do not involve textual conflicts at all (the
> important implication of which is that 'rerere' fundamentally will
> not trigger to help resolving them) [*1*], and are used to create
> evil merge when a corresponding branch is merged to 'pu' (and down).

I see. My question was unclear, I agree. Please let me re-try:

So essentially, what your cherry-pick'able commits are is a way to store
what rerere would have stored (i.e. the set of merge conflicts together
with their resolution)?

If so, what tooling do you have to identify quickly what to cherry-pick,
given merge conflicts?

(I know I could spend some half hour to scour your `todo` branch and the
documentation you wrote about how you maintain Git, but you already know
the answer to this question, and it would probably be interesting to
others who are as eager to have better tools for handling merge conflicts
as I am, too, so it would save time overall to discuss this single
question here.)

> *1* One topic adds an extra parameter to read_index_from() that has
> been and still is defined in a file and merged to 'pu' first, while
> another topic adds a new callsite for the same function in a file
> that the former topic does not touch, hence a merge of the latter
> topic has no textual conflict to the file with a new callsite, but
> still needs adjusting.  That sort of think.

Okay, so it is even more intricate than `rerere`: it does not only store
the merge conflicts (by grace of using the "patch ID" of the merge
conflicts) together with their resolution, but instead it has some sort of
idea of what context needs to be met to require the resolution?

Color me intrigued.

If you really found a way to automate describing, say, that a function
signature changed in one branch, and a caller was introduced in another,
and that merging those requires adjusting the caller in a specific way, I
now *really* think that this should be made available more broadly.

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

On 06/03/2018 12:45, Sergey Organov wrote:
> 
> > > The only thing I wonder of here is how would we check if the 
> > > "rebased" merge M' was "clean", or should we stop for user amendment? 
> > > With that other approach Sergey described, we have U1'==U2' to test with.
> >
> > I think (though I haven't rigorously proved to myself) that in the
> > absence of conflicts this scheme has well defined semantics (the merges
> > can be commuted), so the result should be predicable from the users
> > point of view so maybe it could just offer an option to stop.
> 
> Yes, hopefully it's predictable, but is it the intended one? We don't
> know, so there is still some level of uncertainty.
> 
> When in doubt, I try to find similar cases. There are two I'm aware of:
> 
> 1. "git merge" just commits the result when there are no conflicts.
> However, it supposedly has been run by the user just now, and thus user
> can amend what he gets. That's effectively a stop for amendment from our
> POV.
> 
> 2. When rebasing, "rerere", when fires, stages the changes, and rebasing
> stops for amendment. For me "rerere" behavior is rather annoying (I've
> never in fact amended what it prepared), but I always assumed there are
> good reasons it behaves this way.
> 
> Overall, to be consistent, it seems we do need to stop at U1' != U2', at
> least by default. Additional options could be supported then to specify
> user intentions, both on the command level and in the todo list,
> provided it proves to be useful.

Just to say I agree with this, `if U1' == U2' then proceed else stop` 
seems as a good sanity check.

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Phillip Wood]

On 06/03/18 18:12, Johannes Schindelin wrote:
> Hi Phillip,
> 
> On Tue, 6 Mar 2018, Phillip Wood wrote:
> 
>> On 03/03/18 00:29, Igor Djordjevic wrote:
>>>
>>> On 02/03/2018 12:31, Phillip Wood wrote:

> Thinking about it overnight, I now suspect that original proposal
> had a mistake in the final merge step. I think that what you did is
> a way to fix it, and I want to try to figure what exactly was wrong
> in the original proposal and to find simpler way of doing it right.
>
> The likely solution is to use original UM as a merge-base for final
> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty
> natural though, as that's exactly UM from which both U1' and U2'
> have diverged due to rebasing and other history editing.

 Hi Sergey, I've been following this discussion from the sidelines,
 though I haven't had time to study all the posts in this thread in
 detail. I wonder if it would be helpful to think of rebasing a merge
 as merging the changes in the parents due to the rebase back into the
 original merge. So for a merge M with parents A B C that are rebased
 to A' B' C' the rebased merge M' would be constructed by (ignoring
 shell quoting issues)

 git checkout --detach M
 git merge-recursive A -- M A'
 tree=$(git write-tree)
 git merge-recursive B -- $tree B'
 tree=$(git write-tree)
 git merge-recursive C -- $tree C'
 tree=$(git write-tree)
 M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')

 This should pull in all the changes from the parents while preserving
 any evil conflict resolution in the original merge. It superficially
 reminds me of incremental merging [1] but it's so long since I looked at
 that I'm not sure if there are any significant similarities.

 [1] https://github.com/mhagger/git-imerge
>>>
>>> Interesting, from quick test[3], this seems to produce the same 
>>> result as that other test I previously provided[2], where temporary 
>>> commits U1' and U2' are finally merged with original M as a base :)
>>>
>>> Just that this looks like even more straight-forward approach...?
>>>
>>> The only thing I wonder of here is how would we check if the 
>>> "rebased" merge M' was "clean", or should we stop for user amendment? 
>>> With that other approach Sergey described, we have U1'==U2' to test with.
>>
>> I think (though I haven't rigorously proved to myself) that in the
>> absence of conflicts this scheme has well defined semantics (the merges
>> can be commuted), so the result should be predicable from the users
>> point of view so maybe it could just offer an option to stop.
> 
> I am not so sure that the result is independent of the order of the
> merges. In other words, I am not necessarily certain that it is impossible
> to concoct A,A',B,B' commits where merging B'/B before A'/A has a
> different result than merging A'/A before B'/B.
> 
> Remember, when constructing counter-examples to hypotheses, those
> counter-examples do not really *have* to make sense on their own. For
> example, A' could introduce *completely different* changes from A, and the
> same is true for B' and B.
> 
> I could imagine, for example, that using a ton of consecutive empty lines,
> and using patches that insert something into these empty lines (and are
> thusly inherently ambiguous when said set of empty lines has changed),
> could even introduce a merge conflict in one order, but no conflict in the
> other.

Yes I should have thought of that given I've just been working on making
'add -p' more robust when there are lots of identical lines.

> Even so, I think that merging in the order of the parents makes the most
> sense, and that using that strategy makes sense, too, because you really
> have to try hard to make it fail.
> 
> Ciao,
> Dscho
> 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Junio C Hamano]

Johannes Schindelin  writes:

>> Non-textual semantic conflicts are made (in the best case just once)
>> as a separate commit on top of mechanical auto-merge whose focus is
>> predictability (rather than cleverness) done by Git, and then that
>> separate commit is kept outside the history.  When replaying these
>> merges to rebuild the 'pu' branch, after resetting the tip to
>> 'master', each topic is merged mechanically, and if such a fix-up
>> commit is present, "cherry-pick --no-commit" applies it and then
>> "commit --amend --no-edit" to adjust the merge.  I find it quite
>> valuable to have a separate record of what "evil" non-mechanical
>> adjustment was done, which I know won't be lost in the noise when
>> these merges need to be redone daily or more often.
>
> So essentially, you have something that `git rerere` would have learned,
> but as a commit?

You probably wouldn't be asking that if you read what you cut out
when you quoted above ;-) 

There are a collection of cherry-pickable commits in hierarchy under
refs/merge-fix.  They are indexed by the branch that will cause
semantic conflicts that do not involve textual conflicts at all (the
important implication of which is that 'rerere' fundamentally will
not trigger to help resolving them) [*1*], and are used to create
evil merge when a corresponding branch is merged to 'pu' (and down).

[Footnote]

*1* One topic adds an extra parameter to read_index_from() that has
been and still is defined in a file and merged to 'pu' first, while
another topic adds a new callsite for the same function in a file
that the former topic does not touch, hence a merge of the latter
topic has no textual conflict to the file with a new callsite, but
still needs adjusting.  That sort of think.

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Junio,

On Tue, 6 Mar 2018, Junio C Hamano wrote:

> Phillip Wood  writes:
> 
> > I wonder if just having a predicable result rather than forcing the
> > rebase to stop if the user just squashes a fixup commit into a topic
> > branch that is the parent of a merge might be more convenient in
> > practice.
> 
> Unless I am misunderstanding what you are saying, that is pretty much
> what I have automated for my daily rebuild of the 'pu' branch
> 
> Non-textual semantic conflicts are made (in the best case just once)
> as a separate commit on top of mechanical auto-merge whose focus is
> predictability (rather than cleverness) done by Git, and then that
> separate commit is kept outside the history.  When replaying these
> merges to rebuild the 'pu' branch, after resetting the tip to
> 'master', each topic is merged mechanically, and if such a fix-up
> commit is present, "cherry-pick --no-commit" applies it and then
> "commit --amend --no-edit" to adjust the merge.  I find it quite
> valuable to have a separate record of what "evil" non-mechanical
> adjustment was done, which I know won't be lost in the noise when
> these merges need to be redone daily or more often.

So essentially, you have something that `git rerere` would have learned,
but as a commit?

Might make sense to make that procedure more accessible to others, too ;-)
I know that *I* would use it.

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Sergey,

On Wed, 7 Mar 2018, Sergey Organov wrote:

> Johannes Schindelin  writes:
> 
> > How can your approach -- which relies *very much* on having the
> > original parent commits -- not *require* that consistency check?
> 
> I don't understand what you mean, sorry. Could you please point me to
> the *require* you talk about in the original proposal?

Imagine a todo list that contains this line

merge -C abcdef 123456

and now the user edits it (this is an interactive rebase, after all),
adding another merge head:

merge -C abcdef 987654 123456

Now your strategy would have a serious problem: to find the original
version of 987654. If there was one.

> > What would your approach (that still has no satisfyingly trivial
> > explanation, in my mind)
> 
> Here is one-liner: rebase sides of the merge commit and then 3-way
> merge them, using original merge commit as merge base.

But I already pointed out how that would undo a commit having been
dropped.

> > do if somebody edited a `merge` command and let it merge a completely
> > unrelated commit?
> 
> Don't see a problem, sorry. The method should still work, provided you have
> original merge commit and two new parents for the new merge.

That is assuming a lot. That is exactly what this consistency check is
for, that I mentioned earlier, and which you listed as a downside of
Phillip's strategy (forgetting that your strategy has the same downside,
so...).

But I guess that you are still talking about the non-interactive version
of the rebase, and missed that our conversation proceeded to the point
where we want that same strategy to work *also* in the interactive version
(and not have a completely different functionality depending whether you
use --interactive or not)?

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Johannes Schindelin  writes:

> Hi Sergey,
>
> On Wed, 7 Mar 2018, Sergey Organov wrote:
>
>> Johannes Schindelin  writes:
>> 
>> > On Tue, 6 Mar 2018, Phillip Wood wrote:
>> >
>> >> On 03/03/18 00:29, Igor Djordjevic wrote:
>> >> > 
>> >> > On 02/03/2018 12:31, Phillip Wood wrote:
>> >> >>
>> >> >>> Thinking about it overnight, I now suspect that original proposal
>> >> >>> had a mistake in the final merge step. I think that what you did is
>> >> >>> a way to fix it, and I want to try to figure what exactly was wrong
>> >> >>> in the original proposal and to find simpler way of doing it right.
>> >> >>>
>> >> >>> The likely solution is to use original UM as a merge-base for final
>> >> >>> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty
>> >> >>> natural though, as that's exactly UM from which both U1' and U2'
>> >> >>> have diverged due to rebasing and other history editing.
>> >> >>
>> >> >> Hi Sergey, I've been following this discussion from the sidelines,
>> >> >> though I haven't had time to study all the posts in this thread in
>> >> >> detail. I wonder if it would be helpful to think of rebasing a merge
>> >> >> as merging the changes in the parents due to the rebase back into the
>> >> >> original merge. So for a merge M with parents A B C that are rebased
>> >> >> to A' B' C' the rebased merge M' would be constructed by (ignoring
>> >> >> shell quoting issues)
>> >> >>
>> >> >> git checkout --detach M
>> >> >> git merge-recursive A -- M A'
>> >> >> tree=$(git write-tree)
>> >> >> git merge-recursive B -- $tree B'
>> >> >> tree=$(git write-tree)
>> >> >> git merge-recursive C -- $tree C'
>> >> >> tree=$(git write-tree)
>> >> >> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
>> >> >>
>> >> >> This should pull in all the changes from the parents while preserving
>> >> >> any evil conflict resolution in the original merge. It superficially
>> >> >> reminds me of incremental merging [1] but it's so long since I looked 
>> >> >> at
>> >> >> that I'm not sure if there are any significant similarities.
>> >> >>
>> >> >> [1] https://github.com/mhagger/git-imerge
>> >> > 
>> >> > Interesting, from quick test[3], this seems to produce the same 
>> >> > result as that other test I previously provided[2], where temporary 
>> >> > commits U1' and U2' are finally merged with original M as a base :)
>> >> > 
>> >> > Just that this looks like even more straight-forward approach...?
>> >> > 
>> >> > The only thing I wonder of here is how would we check if the 
>> >> > "rebased" merge M' was "clean", or should we stop for user amendment? 
>> >> > With that other approach Sergey described, we have U1'==U2' to test 
>> >> > with.
>> >> 
>> >> I think (though I haven't rigorously proved to myself) that in the
>> >> absence of conflicts this scheme has well defined semantics (the merges
>> >> can be commuted), so the result should be predicable from the users
>> >> point of view so maybe it could just offer an option to stop.
>> >
>> > I am not so sure that the result is independent of the order of the
>> > merges. In other words, I am not necessarily certain that it is impossible
>> > to concoct A,A',B,B' commits where merging B'/B before A'/A has a
>> > different result than merging A'/A before B'/B.
>> >
>> > Remember, when constructing counter-examples to hypotheses, those
>> > counter-examples do not really *have* to make sense on their own. For
>> > example, A' could introduce *completely different* changes from A, and the
>> > same is true for B' and B.
>> >
>> > I could imagine, for example, that using a ton of consecutive empty lines,
>> > and using patches that insert something into these empty lines (and are
>> > thusly inherently ambiguous when said set of empty lines has changed),
>> > could even introduce a merge conflict in one order, but no conflict in the
>> > other.
>> >
>> > Even so, I think that merging in the order of the parents makes the most
>> > sense, and that using that strategy makes sense, too, because you really
>> > have to try hard to make it fail.
>> 
>> Alternatively, consider to adopt the original approach that has none of
>> these issues as it uses exactly the same method for rebasing merge
>> commits that you are already using for rebasing simple commits, not to
>> mention the advantage of the built-in consistency check.
>
> Surely I misunderstand?
>
> How can your approach -- which relies *very much* on having the original
> parent commits -- not *require* that consistency check?

I don't understand what you mean, sorry. Could you please point me
to the *require* you talk about in the original proposal?

> What would your approach (that still has no satisfyingly trivial
> explanation, in my mind)

Here is one-liner: rebase sides of the merge commit and then 3-way
merge them, using original merge commit as merge base.

> do if somebody edited a `merge` command and let it merge a completely
> 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Buga,

On Wed, 7 Mar 2018, Igor Djordjevic wrote:

> On 05/03/2018 18:29, Johannes Schindelin wrote:
> > 
> > > By the way, is there documentation for `git merge-recursive`
> > > anywhere, besides the code itself...? :$
> > 
> > I am not aware of any. The commit message adding the command is not very
> > illuminating (https://github.com/git-for-windows/git/commit/720d150c4):
> > 
> > Add a new merge strategy by Fredrik Kuivinen.
> > 
> > I really wanted to try this out, instead of asking for an adjustment
> > to the 'git merge' driver and waiting.  For now the new strategy is
> > called 'fredrik' and not in the list of default strategies to be tried.
> > 
> > The script wants Python 2.4 so this commit also adjusts Debian and RPM
> > build procecure files.
> > 
> > Digging through https://public-inbox.org/git/ during that time frame comes
> > up with this hit, though:
> > 
> > https://public-inbox.org/git/20050907164734.ga20...@c165.ib.student.liu.se/
> > 
> > which is still not a good documentation of the algorithm. You can probably
> > dig further yourself, but I think I can describe it very quickly here:
> > 
> > To merge two commits recursively, you first have to find their "merge
> > bases". If there was an obvious branch point, then that is the merge base.
> > But when you start a branch off of master, then work a bit, then merge
> > master, you already have two merge bases.
> > 
> > The trick about the recursive merge is to reduce the number of merge bases
> > iteratively to one. It does that by taking two merge bases, and performing
> > a recursive merge on them, which generates a "virtual" commit, the
> > condensed merge base. That one is then merged recursively with the next
> > merge base, until there is only one left.
> > 
> > A recursive merge of two commits with exactly one merge base is simply a
> > three-way merge.
> > 
> > I vaguely remember that there was something funny about the order in which
> > order you want to process the merge bases: if you did it in one
> > (chronological) direction, it worked beautifully, in the other direction
> > it would generate tons of merge conflicts or something like that.
> 
> Thanks, this is very informative (together with the linked discussion).
> 
> Not remembering seeing this before, I wasn`t really sure if this was 
> some undocumented core Git (plumbing?) utility (used withing Git 
> itself, too), or just a leftover (yet still useful) sample tool.

You raise a good point. The "recursive merge" is not really documented
properly. Maybe you can enhance my vague description and provide a patch
for Documentation/technical/?

Thanks,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Sergey,

On Wed, 7 Mar 2018, Sergey Organov wrote:

> Johannes Schindelin  writes:
> 
> > On Tue, 6 Mar 2018, Phillip Wood wrote:
> >
> >> On 03/03/18 00:29, Igor Djordjevic wrote:
> >> > 
> >> > On 02/03/2018 12:31, Phillip Wood wrote:
> >> >>
> >> >>> Thinking about it overnight, I now suspect that original proposal
> >> >>> had a mistake in the final merge step. I think that what you did is
> >> >>> a way to fix it, and I want to try to figure what exactly was wrong
> >> >>> in the original proposal and to find simpler way of doing it right.
> >> >>>
> >> >>> The likely solution is to use original UM as a merge-base for final
> >> >>> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty
> >> >>> natural though, as that's exactly UM from which both U1' and U2'
> >> >>> have diverged due to rebasing and other history editing.
> >> >>
> >> >> Hi Sergey, I've been following this discussion from the sidelines,
> >> >> though I haven't had time to study all the posts in this thread in
> >> >> detail. I wonder if it would be helpful to think of rebasing a merge
> >> >> as merging the changes in the parents due to the rebase back into the
> >> >> original merge. So for a merge M with parents A B C that are rebased
> >> >> to A' B' C' the rebased merge M' would be constructed by (ignoring
> >> >> shell quoting issues)
> >> >>
> >> >> git checkout --detach M
> >> >> git merge-recursive A -- M A'
> >> >> tree=$(git write-tree)
> >> >> git merge-recursive B -- $tree B'
> >> >> tree=$(git write-tree)
> >> >> git merge-recursive C -- $tree C'
> >> >> tree=$(git write-tree)
> >> >> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
> >> >>
> >> >> This should pull in all the changes from the parents while preserving
> >> >> any evil conflict resolution in the original merge. It superficially
> >> >> reminds me of incremental merging [1] but it's so long since I looked at
> >> >> that I'm not sure if there are any significant similarities.
> >> >>
> >> >> [1] https://github.com/mhagger/git-imerge
> >> > 
> >> > Interesting, from quick test[3], this seems to produce the same 
> >> > result as that other test I previously provided[2], where temporary 
> >> > commits U1' and U2' are finally merged with original M as a base :)
> >> > 
> >> > Just that this looks like even more straight-forward approach...?
> >> > 
> >> > The only thing I wonder of here is how would we check if the 
> >> > "rebased" merge M' was "clean", or should we stop for user amendment? 
> >> > With that other approach Sergey described, we have U1'==U2' to test with.
> >> 
> >> I think (though I haven't rigorously proved to myself) that in the
> >> absence of conflicts this scheme has well defined semantics (the merges
> >> can be commuted), so the result should be predicable from the users
> >> point of view so maybe it could just offer an option to stop.
> >
> > I am not so sure that the result is independent of the order of the
> > merges. In other words, I am not necessarily certain that it is impossible
> > to concoct A,A',B,B' commits where merging B'/B before A'/A has a
> > different result than merging A'/A before B'/B.
> >
> > Remember, when constructing counter-examples to hypotheses, those
> > counter-examples do not really *have* to make sense on their own. For
> > example, A' could introduce *completely different* changes from A, and the
> > same is true for B' and B.
> >
> > I could imagine, for example, that using a ton of consecutive empty lines,
> > and using patches that insert something into these empty lines (and are
> > thusly inherently ambiguous when said set of empty lines has changed),
> > could even introduce a merge conflict in one order, but no conflict in the
> > other.
> >
> > Even so, I think that merging in the order of the parents makes the most
> > sense, and that using that strategy makes sense, too, because you really
> > have to try hard to make it fail.
> 
> Alternatively, consider to adopt the original approach that has none of
> these issues as it uses exactly the same method for rebasing merge
> commits that you are already using for rebasing simple commits, not to
> mention the advantage of the built-in consistency check.

Surely I misunderstand?

How can your approach -- which relies *very much* on having the original
parent commits -- not *require* that consistency check?

What would your approach (that still has no satisfyingly trivial
explanation, in my mind) do if somebody edited a `merge` command and let
it merge a completely unrelated commit?

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin  writes:

> Hi Phillip,
>
> On Tue, 6 Mar 2018, Phillip Wood wrote:
>
>> On 03/03/18 00:29, Igor Djordjevic wrote:
>> > 
>> > On 02/03/2018 12:31, Phillip Wood wrote:
>> >>
>> >>> Thinking about it overnight, I now suspect that original proposal
>> >>> had a mistake in the final merge step. I think that what you did is
>> >>> a way to fix it, and I want to try to figure what exactly was wrong
>> >>> in the original proposal and to find simpler way of doing it right.
>> >>>
>> >>> The likely solution is to use original UM as a merge-base for final
>> >>> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty
>> >>> natural though, as that's exactly UM from which both U1' and U2'
>> >>> have diverged due to rebasing and other history editing.
>> >>
>> >> Hi Sergey, I've been following this discussion from the sidelines,
>> >> though I haven't had time to study all the posts in this thread in
>> >> detail. I wonder if it would be helpful to think of rebasing a merge
>> >> as merging the changes in the parents due to the rebase back into the
>> >> original merge. So for a merge M with parents A B C that are rebased
>> >> to A' B' C' the rebased merge M' would be constructed by (ignoring
>> >> shell quoting issues)
>> >>
>> >> git checkout --detach M
>> >> git merge-recursive A -- M A'
>> >> tree=$(git write-tree)
>> >> git merge-recursive B -- $tree B'
>> >> tree=$(git write-tree)
>> >> git merge-recursive C -- $tree C'
>> >> tree=$(git write-tree)
>> >> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
>> >>
>> >> This should pull in all the changes from the parents while preserving
>> >> any evil conflict resolution in the original merge. It superficially
>> >> reminds me of incremental merging [1] but it's so long since I looked at
>> >> that I'm not sure if there are any significant similarities.
>> >>
>> >> [1] https://github.com/mhagger/git-imerge
>> > 
>> > Interesting, from quick test[3], this seems to produce the same 
>> > result as that other test I previously provided[2], where temporary 
>> > commits U1' and U2' are finally merged with original M as a base :)
>> > 
>> > Just that this looks like even more straight-forward approach...?
>> > 
>> > The only thing I wonder of here is how would we check if the 
>> > "rebased" merge M' was "clean", or should we stop for user amendment? 
>> > With that other approach Sergey described, we have U1'==U2' to test with.
>> 
>> I think (though I haven't rigorously proved to myself) that in the
>> absence of conflicts this scheme has well defined semantics (the merges
>> can be commuted), so the result should be predicable from the users
>> point of view so maybe it could just offer an option to stop.
>
> I am not so sure that the result is independent of the order of the
> merges. In other words, I am not necessarily certain that it is impossible
> to concoct A,A',B,B' commits where merging B'/B before A'/A has a
> different result than merging A'/A before B'/B.
>
> Remember, when constructing counter-examples to hypotheses, those
> counter-examples do not really *have* to make sense on their own. For
> example, A' could introduce *completely different* changes from A, and the
> same is true for B' and B.
>
> I could imagine, for example, that using a ton of consecutive empty lines,
> and using patches that insert something into these empty lines (and are
> thusly inherently ambiguous when said set of empty lines has changed),
> could even introduce a merge conflict in one order, but no conflict in the
> other.
>
> Even so, I think that merging in the order of the parents makes the most
> sense, and that using that strategy makes sense, too, because you really
> have to try hard to make it fail.

Alternatively, consider to adopt the original approach that has none of
these issues as it uses exactly the same method for rebasing merge
commits that you are already using for rebasing simple commits, not to
mention the advantage of the built-in consistency check.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Johannes,

On 05/03/2018 18:29, Johannes Schindelin wrote:
> 
> > By the way, is there documentation for `git merge-recursive` 
> > anywhere, besides the code itself...? :$
> 
> I am not aware of any. The commit message adding the command is not very
> illuminating (https://github.com/git-for-windows/git/commit/720d150c4):
> 
> Add a new merge strategy by Fredrik Kuivinen.
> 
> I really wanted to try this out, instead of asking for an adjustment
> to the 'git merge' driver and waiting.  For now the new strategy is
> called 'fredrik' and not in the list of default strategies to be tried.
> 
> The script wants Python 2.4 so this commit also adjusts Debian and RPM
> build procecure files.
> 
> Digging through https://public-inbox.org/git/ during that time frame comes
> up with this hit, though:
> 
> https://public-inbox.org/git/20050907164734.ga20...@c165.ib.student.liu.se/
> 
> which is still not a good documentation of the algorithm. You can probably
> dig further yourself, but I think I can describe it very quickly here:
> 
> To merge two commits recursively, you first have to find their "merge
> bases". If there was an obvious branch point, then that is the merge base.
> But when you start a branch off of master, then work a bit, then merge
> master, you already have two merge bases.
> 
> The trick about the recursive merge is to reduce the number of merge bases
> iteratively to one. It does that by taking two merge bases, and performing
> a recursive merge on them, which generates a "virtual" commit, the
> condensed merge base. That one is then merged recursively with the next
> merge base, until there is only one left.
> 
> A recursive merge of two commits with exactly one merge base is simply a
> three-way merge.
> 
> I vaguely remember that there was something funny about the order in which
> order you want to process the merge bases: if you did it in one
> (chronological) direction, it worked beautifully, in the other direction
> it would generate tons of merge conflicts or something like that.

Thanks, this is very informative (together with the linked discussion).

Not remembering seeing this before, I wasn`t really sure if this was 
some undocumented core Git (plumbing?) utility (used withing Git 
itself, too), or just a leftover (yet still useful) sample tool.

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Phillip,

On Tue, 6 Mar 2018, Phillip Wood wrote:

> On 03/03/18 00:29, Igor Djordjevic wrote:
> > 
> > On 02/03/2018 12:31, Phillip Wood wrote:
> >>
> >>> Thinking about it overnight, I now suspect that original proposal
> >>> had a mistake in the final merge step. I think that what you did is
> >>> a way to fix it, and I want to try to figure what exactly was wrong
> >>> in the original proposal and to find simpler way of doing it right.
> >>>
> >>> The likely solution is to use original UM as a merge-base for final
> >>> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty
> >>> natural though, as that's exactly UM from which both U1' and U2'
> >>> have diverged due to rebasing and other history editing.
> >>
> >> Hi Sergey, I've been following this discussion from the sidelines,
> >> though I haven't had time to study all the posts in this thread in
> >> detail. I wonder if it would be helpful to think of rebasing a merge
> >> as merging the changes in the parents due to the rebase back into the
> >> original merge. So for a merge M with parents A B C that are rebased
> >> to A' B' C' the rebased merge M' would be constructed by (ignoring
> >> shell quoting issues)
> >>
> >> git checkout --detach M
> >> git merge-recursive A -- M A'
> >> tree=$(git write-tree)
> >> git merge-recursive B -- $tree B'
> >> tree=$(git write-tree)
> >> git merge-recursive C -- $tree C'
> >> tree=$(git write-tree)
> >> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
> >>
> >> This should pull in all the changes from the parents while preserving
> >> any evil conflict resolution in the original merge. It superficially
> >> reminds me of incremental merging [1] but it's so long since I looked at
> >> that I'm not sure if there are any significant similarities.
> >>
> >> [1] https://github.com/mhagger/git-imerge
> > 
> > Interesting, from quick test[3], this seems to produce the same 
> > result as that other test I previously provided[2], where temporary 
> > commits U1' and U2' are finally merged with original M as a base :)
> > 
> > Just that this looks like even more straight-forward approach...?
> > 
> > The only thing I wonder of here is how would we check if the 
> > "rebased" merge M' was "clean", or should we stop for user amendment? 
> > With that other approach Sergey described, we have U1'==U2' to test with.
> 
> I think (though I haven't rigorously proved to myself) that in the
> absence of conflicts this scheme has well defined semantics (the merges
> can be commuted), so the result should be predicable from the users
> point of view so maybe it could just offer an option to stop.

I am not so sure that the result is independent of the order of the
merges. In other words, I am not necessarily certain that it is impossible
to concoct A,A',B,B' commits where merging B'/B before A'/A has a
different result than merging A'/A before B'/B.

Remember, when constructing counter-examples to hypotheses, those
counter-examples do not really *have* to make sense on their own. For
example, A' could introduce *completely different* changes from A, and the
same is true for B' and B.

I could imagine, for example, that using a ton of consecutive empty lines,
and using patches that insert something into these empty lines (and are
thusly inherently ambiguous when said set of empty lines has changed),
could even introduce a merge conflict in one order, but no conflict in the
other.

Even so, I think that merging in the order of the parents makes the most
sense, and that using that strategy makes sense, too, because you really
have to try hard to make it fail.

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Junio C Hamano]

Phillip Wood  writes:

> I wonder if just having a predicable result rather than forcing the
> rebase to stop if the user just squashes a fixup commit into a topic
> branch that is the parent of a merge might be more convenient in practice.

Unless I am misunderstanding what you are saying, that is pretty
much what I have automated for my daily rebuild of the 'pu' branch

Non-textual semantic conflicts are made (in the best case just once)
as a separate commit on top of mechanical auto-merge whose focus is
predictability (rather than cleverness) done by Git, and then that
separate commit is kept outside the history.  When replaying these
merges to rebuild the 'pu' branch, after resetting the tip to
'master', each topic is merged mechanically, and if such a fix-up
commit is present, "cherry-pick --no-commit" applies it and then
"commit --amend --no-edit" to adjust the merge.  I find it quite
valuable to have a separate record of what "evil" non-mechanical
adjustment was done, which I know won't be lost in the noise when
these merges need to be redone daily or more often.

The Appendix in Documentation/howto/maintain-git.txt talks about
this process.  You can see what topics have such merge-fix defined
by peeking https://github.com/gitster/git/ repository.

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Phillip,

Phillip Wood  writes:

> On 03/03/18 00:29, Igor Djordjevic wrote:
>> Hi Phillip,

[...]

>> 
>> The only thing I wonder of here is how would we check if the 
>> "rebased" merge M' was "clean", or should we stop for user amendment? 
>> With that other approach Sergey described, we have U1'==U2' to test with.
>
> I think (though I haven't rigorously proved to myself) that in the
> absence of conflicts this scheme has well defined semantics (the merges
> can be commuted), so the result should be predicable from the users
> point of view so maybe it could just offer an option to stop.

Yes, hopefully it's predictable, but is it the intended one? We don't
know, so there is still some level of uncertainty.

When in doubt, I try to find similar cases. There are two I'm aware of:

1. "git merge" just commits the result when there are no conflicts.
However, it supposedly has been run by the user just now, and thus user
can amend what he gets. That's effectively a stop for amendment from our
POV.

2. When rebasing, "rerere", when fires, stages the changes, and rebasing
stops for amendment. For me "rerere" behavior is rather annoying (I've
never in fact amended what it prepared), but I always assumed there are
good reasons it behaves this way.

Overall, to be consistent, it seems we do need to stop at U1' != U2', at
least by default. Additional options could be supported then to specify
user intentions, both on the command level and in the todo list,
provided it proves to be useful.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Phillip Wood]

On 05/03/18 05:00, Sergey Organov wrote:
> Hi Plillip and Igor,
> 
> Igor Djordjevic  writes:
>> Hi Phillip,
>>
>> On 02/03/2018 12:31, Phillip Wood wrote:
>>>
 Thinking about it overnight, I now suspect that original proposal had a
 mistake in the final merge step. I think that what you did is a way to
 fix it, and I want to try to figure what exactly was wrong in the
 original proposal and to find simpler way of doing it right.

 The likely solution is to use original UM as a merge-base for final
 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
 though, as that's exactly UM from which both U1' and U2' have diverged
 due to rebasing and other history editing.
>>>
>>> Hi Sergey, I've been following this discussion from the sidelines,
>>> though I haven't had time to study all the posts in this thread in
>>> detail. I wonder if it would be helpful to think of rebasing a merge as
>>> merging the changes in the parents due to the rebase back into the
>>> original merge. So for a merge M with parents A B C that are rebased to
>>> A' B' C' the rebased merge M' would be constructed by (ignoring shell
>>> quoting issues)
>>>
>>> git checkout --detach M
>>> git merge-recursive A -- M A'
>>> tree=$(git write-tree)
>>> git merge-recursive B -- $tree B'
>>> tree=$(git write-tree)
>>> git merge-recursive C -- $tree C'
>>> tree=$(git write-tree)
>>> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
>>>
>>> This should pull in all the changes from the parents while preserving
>>> any evil conflict resolution in the original merge. It superficially
>>> reminds me of incremental merging [1] but it's so long since I looked at
>>> that I'm not sure if there are any significant similarities.
>>>
>>> [1] https://github.com/mhagger/git-imerge
>>
>> Interesting, from quick test[3], this seems to produce the same 
>> result as that other test I previously provided[2], where temporary 
>> commits U1' and U2' are finally merged with original M as a base :)
> 
> Looks like sound approach and it's interesting if these 2 methods do in
> fact always bring the same result. Because if we look at the (now fixed)
> original approach closely, it also just gathers the changes in merge
> parents into U1' and U2', then merges the changes back into the original
> M (=U1=U2=UM).
> 
> Overall, this one looks like another implementation of essentially the
> same method and confirms that we all have the right thought direction
> here.
> 

Yes, I think they are doing the same thing. If there are no conflicts
then U1' is the should as "git merge-recursive A -- M A'". My patch
algebra isn't very good, but I think one ought to be able to show that
in the absence of conflicts the two approaches are equivalent.

>>
>> Just that this looks like even more straight-forward approach...?
>>
>> The only thing I wonder of here is how would we check if the 
>> "rebased" merge M' was "clean", or should we stop for user amendment? 
>> With that other approach Sergey described, we have U1'==U2' to test
>> with.
> 
> That's an advantage of the original, yes.

I wonder if just having a predicable result rather than forcing the
rebase to stop if the user just squashes a fixup commit into a topic
branch that is the parent of a merge might be more convenient in practice.

Best Wishes

Phillip

> -- Sergey
> 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Phillip Wood]

On 03/03/18 00:29, Igor Djordjevic wrote:
> Hi Phillip,
> 
> On 02/03/2018 12:31, Phillip Wood wrote:
>>
>>> Thinking about it overnight, I now suspect that original proposal had a
>>> mistake in the final merge step. I think that what you did is a way to
>>> fix it, and I want to try to figure what exactly was wrong in the
>>> original proposal and to find simpler way of doing it right.
>>>
>>> The likely solution is to use original UM as a merge-base for final
>>> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
>>> though, as that's exactly UM from which both U1' and U2' have diverged
>>> due to rebasing and other history editing.
>>
>> Hi Sergey, I've been following this discussion from the sidelines,
>> though I haven't had time to study all the posts in this thread in
>> detail. I wonder if it would be helpful to think of rebasing a merge as
>> merging the changes in the parents due to the rebase back into the
>> original merge. So for a merge M with parents A B C that are rebased to
>> A' B' C' the rebased merge M' would be constructed by (ignoring shell
>> quoting issues)
>>
>> git checkout --detach M
>> git merge-recursive A -- M A'
>> tree=$(git write-tree)
>> git merge-recursive B -- $tree B'
>> tree=$(git write-tree)
>> git merge-recursive C -- $tree C'
>> tree=$(git write-tree)
>> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
>>
>> This should pull in all the changes from the parents while preserving
>> any evil conflict resolution in the original merge. It superficially
>> reminds me of incremental merging [1] but it's so long since I looked at
>> that I'm not sure if there are any significant similarities.
>>
>> [1] https://github.com/mhagger/git-imerge
> 
> Interesting, from quick test[3], this seems to produce the same 
> result as that other test I previously provided[2], where temporary 
> commits U1' and U2' are finally merged with original M as a base :)
> 
> Just that this looks like even more straight-forward approach...?
> 
> The only thing I wonder of here is how would we check if the 
> "rebased" merge M' was "clean", or should we stop for user amendment? 
> With that other approach Sergey described, we have U1'==U2' to test with.

I think (though I haven't rigorously proved to myself) that in the
absence of conflicts this scheme has well defined semantics (the merges
can be commuted), so the result should be predicable from the users
point of view so maybe it could just offer an option to stop.

> 
> By the way, is there documentation for `git merge-recursive` 
> anywhere, besides the code itself...? :$

Not that I'm aware of unfortunately. It's pretty simple though

git merge-recursive []  --  

The options are listed on the 'git merge' man page but you specify them
as '--option' rather than '-Xoption'. I'm not sure what the exact
requirements are for the index, having it match  should always
be safe.

> 
> Thanks, Buga
> 
> [2] 
> https://public-inbox.org/git/f1a960dc-cc5c-e7b0-10b6-39e551665...@gmail.com/
> [3] Quick test script:
> -- >8 --
> #!/bin/sh
> 
> # rm -rf ./.git
> # rm -f ./test.txt
> 
> git init
> 
> touch ./test.txt
> git add -- test.txt
> 
> # prepare repository
> for i in {1..8}
> do
>   echo X$i >>test.txt
>   git commit -am "X$i"
> done
> 
> # prepare branch A
> git checkout -b A
> sed -i '2iA1' test.txt
> git commit -am "A1"
> sed -i '4iA2' test.txt
> git commit -am "A2"
> sed -i '6iA3' test.txt
> git commit -am "A3"
> 
> # prepare branch B
> git checkout -b B master
> sed -i '5iB1' test.txt
> git commit -am "B1"
> sed -i '7iB2' test.txt
> git commit -am "B2"
> sed -i '9iB3' test.txt
> git commit -am "B3"
> 
> git checkout -b topic A
> git merge -s ours --no-commit B # merge A and B with `-s ours`
> sed -i '8iM' test.txt   # amend merge commit ("evil merge")
> git commit -am "M"
> git tag M #original-merge
> 
> # master moves on...
> git checkout master
> git cherry-pick B^ # cherry-pick B2 into master
> sed -i "1iX9" test.txt # add X9
> git commit -am "X9"
> 
> # (0) ---X8--B2'--X9 (master)
> #|\
> #| A1---A2---A3 (A)
> #| \
> #|  M (topic)
> #| /
> #\-B1---B2---B3 (B)
> 
> # simple/naive demonstration of proposed merge rebasing logic
> # using iterative merge-recursive, preserving merge commit manual
> # amendments, testing `-s ours` merge with cherry-picking from
> # obsoleted part, but still respecting interactively rebased
> # added/modified/dropped/cherry-picked commits :)
> 
> git checkout -b A-prime A
> git reset --hard HEAD^ # drop A3 from A
> sed -i '/A1/c\A12' test.txt# amend A1 to A12
> git commit -a --amend --no-edit
> git rebase master  # rebase A onto master
> git cherry-pick B  # cherry-pick B3 into A
> 
> git checkout -b B-prime B
> git rebase master  # rebase B onto master
> sed -i '12iB4' test.txt# add B4
> git commit 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Phillip,

On Fri, 2 Mar 2018, Phillip Wood wrote:

> On 01/03/18 05:39, Sergey Organov wrote:
> > 
> > Igor Djordjevic  writes:
> > 
> >> On 28/02/2018 06:19, Sergey Organov wrote:
> >>>
> > (3) ---X1---o---o---o---o---o---X2
> >|\   |\
> >| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
> >| \  |
> >|  M |
> >| /  |
> >\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
> >
> 
>  Meh, I hope I`m rushing it now, but for example, if we had decided to 
>  drop commit A2 during an interactive rebase (so losing A2' from 
>  diagram above), wouldn`t U2' still introduce those changes back, once 
>  U1' and U2' are merged, being incorrect/unwanted behavior...? :/
> >>>
> >>> I think the method will handle this nicely.
> >>
> >> That`s what I thought as well. And then I made a test. And then the 
> >> test broke... Now, might be the test was flawed in the first place, 
> >> but thinking about it a bit more, it does seem to make sense not to 
> >> handle this case nicely :/
> > 
> > Yeah, I now see it myself. I'm sorry for being lazy and not inspecting
> > this more carefully in the first place.
> > 
> > [...]
> > 
> >> So while your original proposal currently seems like it could be 
> >> working nicely for non-interactive rebase (and might be some simpler 
> >> interactive ones), now hitting/acknowledging its first real use 
> >> limit, my additional quick attempt[1] just tries to aid pretty 
> >> interesting case of complicated interactive rebase, too, where we 
> >> might be able to do better as well, still using you original proposal 
> >> as a base idea :)
> > 
> > Yes, thank you for pushing me back to reality! :-) The work and thoughts
> > you are putting into solving the puzzle are greatly appreciated!
> > 
> > Thinking about it overnight, I now suspect that original proposal had a
> > mistake in the final merge step. I think that what you did is a way to
> > fix it, and I want to try to figure what exactly was wrong in the
> > original proposal and to find simpler way of doing it right.
> > 
> > The likely solution is to use original UM as a merge-base for final
> > 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
> > though, as that's exactly UM from which both U1' and U2' have diverged
> > due to rebasing and other history editing.
> 
> Hi Sergey, I've been following this discussion from the sidelines,
> though I haven't had time to study all the posts in this thread in
> detail. I wonder if it would be helpful to think of rebasing a merge as
> merging the changes in the parents due to the rebase back into the
> original merge. So for a merge M with parents A B C that are rebased to
> A' B' C' the rebased merge M' would be constructed by (ignoring shell
> quoting issues)
> 
> git checkout --detach M
> git merge-recursive A -- M A'
> tree=$(git write-tree)
> git merge-recursive B -- $tree B'
> tree=$(git write-tree)
> git merge-recursive C -- $tree C'
> tree=$(git write-tree)
> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')

(The $tree obviously wants to be passed as parameter to commit-tree, but
it's easy to get the idea.)

> This should pull in all the changes from the parents while preserving
> any evil conflict resolution in the original merge. It superficially
> reminds me of incremental merging [1] but it's so long since I looked at
> that I'm not sure if there are any significant similarities.

Interesting.

Basically, this is pretending that A'/B'/C' were not the result of
rebases, but of merges between A/B/C and upstream, and then performing an
octopus merge of M, A', B' and C'.

It is a beautiful application of the duality between merges and rebases:
ideally, they both result in the same tree [*1*].

That is a pretty clean and simple-to-describe paradigm to work off of, and
to reason about.

Ciao,
Dscho

Footnote *1*: I frequently use that duality in work to rebase "clean"
patches on top of upstream, and use that rebased branch to figure out how
to resolve merge conflicts when merging upstream into a long-running
branch (whose tree is identical to the pre-rebase version of the clean
patches). And yes, I *think* that this is essentially Michael Haggerty's
`git-imerge` idea.

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Buga,

On Sat, 3 Mar 2018, Igor Djordjevic wrote:

> By the way, is there documentation for `git merge-recursive` 
> anywhere, besides the code itself...? :$

I am not aware of any. The commit message adding the command is not very
illuminating (https://github.com/git-for-windows/git/commit/720d150c4):

Add a new merge strategy by Fredrik Kuivinen.

I really wanted to try this out, instead of asking for an adjustment
to the 'git merge' driver and waiting.  For now the new strategy is
called 'fredrik' and not in the list of default strategies to be tried.

The script wants Python 2.4 so this commit also adjusts Debian and RPM
build procecure files.

Digging through https://public-inbox.org/git/ during that time frame comes
up with this hit, though:

https://public-inbox.org/git/20050907164734.ga20...@c165.ib.student.liu.se/

which is still not a good documentation of the algorithm. You can probably
dig further yourself, but I think I can describe it very quickly here:

To merge two commits recursively, you first have to find their "merge
bases". If there was an obvious branch point, then that is the merge base.
But when you start a branch off of master, then work a bit, then merge
master, you already have two merge bases.

The trick about the recursive merge is to reduce the number of merge bases
iteratively to one. It does that by taking two merge bases, and performing
a recursive merge on them, which generates a "virtual" commit, the
condensed merge base. That one is then merged recursively with the next
merge base, until there is only one left.

A recursive merge of two commits with exactly one merge base is simply a
three-way merge.

I vaguely remember that there was something funny about the order in which
order you want to process the merge bases: if you did it in one
(chronological) direction, it worked beautifully, in the other direction
it would generate tons of merge conflicts or something like that.

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Plillip and Igor,

Igor Djordjevic  writes:
> Hi Phillip,
>
> On 02/03/2018 12:31, Phillip Wood wrote:
>> 
>> > Thinking about it overnight, I now suspect that original proposal had a
>> > mistake in the final merge step. I think that what you did is a way to
>> > fix it, and I want to try to figure what exactly was wrong in the
>> > original proposal and to find simpler way of doing it right.
>> >
>> > The likely solution is to use original UM as a merge-base for final
>> > 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
>> > though, as that's exactly UM from which both U1' and U2' have diverged
>> > due to rebasing and other history editing.
>> 
>> Hi Sergey, I've been following this discussion from the sidelines,
>> though I haven't had time to study all the posts in this thread in
>> detail. I wonder if it would be helpful to think of rebasing a merge as
>> merging the changes in the parents due to the rebase back into the
>> original merge. So for a merge M with parents A B C that are rebased to
>> A' B' C' the rebased merge M' would be constructed by (ignoring shell
>> quoting issues)
>> 
>> git checkout --detach M
>> git merge-recursive A -- M A'
>> tree=$(git write-tree)
>> git merge-recursive B -- $tree B'
>> tree=$(git write-tree)
>> git merge-recursive C -- $tree C'
>> tree=$(git write-tree)
>> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
>> 
>> This should pull in all the changes from the parents while preserving
>> any evil conflict resolution in the original merge. It superficially
>> reminds me of incremental merging [1] but it's so long since I looked at
>> that I'm not sure if there are any significant similarities.
>> 
>> [1] https://github.com/mhagger/git-imerge
>
> Interesting, from quick test[3], this seems to produce the same 
> result as that other test I previously provided[2], where temporary 
> commits U1' and U2' are finally merged with original M as a base :)

Looks like sound approach and it's interesting if these 2 methods do in
fact always bring the same result. Because if we look at the (now fixed)
original approach closely, it also just gathers the changes in merge
parents into U1' and U2', then merges the changes back into the original
M (=U1=U2=UM).

Overall, this one looks like another implementation of essentially the
same method and confirms that we all have the right thought direction
here.

>
> Just that this looks like even more straight-forward approach...?
>
> The only thing I wonder of here is how would we check if the 
> "rebased" merge M' was "clean", or should we stop for user amendment? 
> With that other approach Sergey described, we have U1'==U2' to test
> with.

That's an advantage of the original, yes.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Phillip,

On 02/03/2018 12:31, Phillip Wood wrote:
> 
> > Thinking about it overnight, I now suspect that original proposal had a
> > mistake in the final merge step. I think that what you did is a way to
> > fix it, and I want to try to figure what exactly was wrong in the
> > original proposal and to find simpler way of doing it right.
> >
> > The likely solution is to use original UM as a merge-base for final
> > 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
> > though, as that's exactly UM from which both U1' and U2' have diverged
> > due to rebasing and other history editing.
> 
> Hi Sergey, I've been following this discussion from the sidelines,
> though I haven't had time to study all the posts in this thread in
> detail. I wonder if it would be helpful to think of rebasing a merge as
> merging the changes in the parents due to the rebase back into the
> original merge. So for a merge M with parents A B C that are rebased to
> A' B' C' the rebased merge M' would be constructed by (ignoring shell
> quoting issues)
> 
> git checkout --detach M
> git merge-recursive A -- M A'
> tree=$(git write-tree)
> git merge-recursive B -- $tree B'
> tree=$(git write-tree)
> git merge-recursive C -- $tree C'
> tree=$(git write-tree)
> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
> 
> This should pull in all the changes from the parents while preserving
> any evil conflict resolution in the original merge. It superficially
> reminds me of incremental merging [1] but it's so long since I looked at
> that I'm not sure if there are any significant similarities.
> 
> [1] https://github.com/mhagger/git-imerge

Interesting, from quick test[3], this seems to produce the same 
result as that other test I previously provided[2], where temporary 
commits U1' and U2' are finally merged with original M as a base :)

Just that this looks like even more straight-forward approach...?

The only thing I wonder of here is how would we check if the 
"rebased" merge M' was "clean", or should we stop for user amendment? 
With that other approach Sergey described, we have U1'==U2' to test with.

By the way, is there documentation for `git merge-recursive` 
anywhere, besides the code itself...? :$

Thanks, Buga

[2] https://public-inbox.org/git/f1a960dc-cc5c-e7b0-10b6-39e551665...@gmail.com/
[3] Quick test script:
-- >8 --
#!/bin/sh

# rm -rf ./.git
# rm -f ./test.txt

git init

touch ./test.txt
git add -- test.txt

# prepare repository
for i in {1..8}
do
echo X$i >>test.txt
git commit -am "X$i"
done

# prepare branch A
git checkout -b A
sed -i '2iA1' test.txt
git commit -am "A1"
sed -i '4iA2' test.txt
git commit -am "A2"
sed -i '6iA3' test.txt
git commit -am "A3"

# prepare branch B
git checkout -b B master
sed -i '5iB1' test.txt
git commit -am "B1"
sed -i '7iB2' test.txt
git commit -am "B2"
sed -i '9iB3' test.txt
git commit -am "B3"

git checkout -b topic A
git merge -s ours --no-commit B # merge A and B with `-s ours`
sed -i '8iM' test.txt   # amend merge commit ("evil merge")
git commit -am "M"
git tag M #original-merge

# master moves on...
git checkout master
git cherry-pick B^ # cherry-pick B2 into master
sed -i "1iX9" test.txt # add X9
git commit -am "X9"

# (0) ---X8--B2'--X9 (master)
#|\
#| A1---A2---A3 (A)
#| \
#|  M (topic)
#| /
#\-B1---B2---B3 (B)

# simple/naive demonstration of proposed merge rebasing logic
# using iterative merge-recursive, preserving merge commit manual
# amendments, testing `-s ours` merge with cherry-picking from
# obsoleted part, but still respecting interactively rebased
# added/modified/dropped/cherry-picked commits :)

git checkout -b A-prime A
git reset --hard HEAD^ # drop A3 from A
sed -i '/A1/c\A12' test.txt# amend A1 to A12
git commit -a --amend --no-edit
git rebase master  # rebase A onto master
git cherry-pick B  # cherry-pick B3 into A

git checkout -b B-prime B
git rebase master  # rebase B onto master
sed -i '12iB4' test.txt# add B4
git commit -am "B4"

git checkout --detach M
git merge-recursive A -- M A-prime
tree="$(git write-tree)"
git merge-recursive B -- $tree B-prime
tree="$(git write-tree)"
git tag M-prime "$(git log --pretty=%B -1 M | git commit-tree $tree -p A-prime 
-p B-prime)"

git update-ref refs/heads/topic "$(git rev-parse M-prime)"
git checkout topic

# (1) ---X8--B2'--X9 (master)
# |\
# | A12--A2'---B3' (A)
# | \
# |  M' (topic)
# | /
# \-B1'--B3'---B4  (B)

# show resulting graph
# echo
# git log --all --decorate --oneline --graph

# in comparison to original merge commit M, rebased merge commit 
# M' is expected to:
#
# - Add X9, from updated "master"
# - Have A1 changed to A12, due to A12 commit 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 02/03/2018 06:40, Sergey Organov wrote:
> 
> > So... In comparison to original merge commit M, rebased merge commit 
> > M' is expected to:
> > 
> >  - Add X9, from updated "master"
> >  - Have A1 changed to A12, due to A12 commit amendment
> >  - Keep A2, rebased as A2'
> >  - Remove A3, due to dropped A3 commit
> >  - Keep amendment from original (evil) merge commit M
> >  - Miss B1' like M does B, due to original `-s ours` merge strategy
> >  - Add B2, cherry-picked as B2' into "master"
> >  - Add B3, cherry-picked as B3' into "A"
> >  - Add B4, added to "B"
> >  - Most important, provide safety mechanism to "fail loud", being 
> >aware of non-trivial things going on, allowing to stop for user 
> >inspection/decision
> > 
> > 
> > There, I hope I didn`t miss any expectation. And, it _seems_ to work 
> > exactly as expected :D
> 
> That's very nice, to the level of being even suspect! :-)

Heh, indeed :) I`m already thinking of some even more complex 
situations. The more use/test cases, the merrier.

Like if number of merge parents (branches) gets changed during 
interactive rebase...

> To avoid falling into euphoria though, we need to keep in mind that
> "expectations" is rather vague concept, and we likely still need to stop
> for user amendment unless we absolutely sure nothing surprising happens.
> I.e., we better require U1'==U2' test to succeed to proceed non-stop
> automatically. Besides, it will be somewhat inline with what 'rerere'
> does.

I totally agree, and I think whatever we come up with, we`ll always 
be missing some deeper context of the original merge, so U1'==U2' 
test is a must - if it fails, even if we didn`t get any conflicts and 
could otherwise proceed automatically, better stop for user sanity check.

I`m still thinking if there could be a situation where test passes, 
but result might still be suspicious (and worth inspecting), though.

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Phillip Wood]

On 01/03/18 05:39, Sergey Organov wrote:
> 
> Hi Igor,
> 
> Igor Djordjevic  writes:
> 
>> Hi Sergey,
>>
>> On 28/02/2018 06:19, Sergey Organov wrote:
>>>
> (3) ---X1---o---o---o---o---o---X2
>|\   |\
>| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
>| \  |
>|  M |
>| /  |
>\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
>

 Meh, I hope I`m rushing it now, but for example, if we had decided to 
 drop commit A2 during an interactive rebase (so losing A2' from 
 diagram above), wouldn`t U2' still introduce those changes back, once 
 U1' and U2' are merged, being incorrect/unwanted behavior...? :/
>>>
>>> I think the method will handle this nicely.
>>
>> That`s what I thought as well. And then I made a test. And then the 
>> test broke... Now, might be the test was flawed in the first place, 
>> but thinking about it a bit more, it does seem to make sense not to 
>> handle this case nicely :/
> 
> Yeah, I now see it myself. I'm sorry for being lazy and not inspecting
> this more carefully in the first place.
> 
> [...]
> 
>> So while your original proposal currently seems like it could be 
>> working nicely for non-interactive rebase (and might be some simpler 
>> interactive ones), now hitting/acknowledging its first real use 
>> limit, my additional quick attempt[1] just tries to aid pretty 
>> interesting case of complicated interactive rebase, too, where we 
>> might be able to do better as well, still using you original proposal 
>> as a base idea :)
> 
> Yes, thank you for pushing me back to reality! :-) The work and thoughts
> you are putting into solving the puzzle are greatly appreciated!
> 
> Thinking about it overnight, I now suspect that original proposal had a
> mistake in the final merge step. I think that what you did is a way to
> fix it, and I want to try to figure what exactly was wrong in the
> original proposal and to find simpler way of doing it right.
> 
> The likely solution is to use original UM as a merge-base for final
> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
> though, as that's exactly UM from which both U1' and U2' have diverged
> due to rebasing and other history editing.

Hi Sergey, I've been following this discussion from the sidelines,
though I haven't had time to study all the posts in this thread in
detail. I wonder if it would be helpful to think of rebasing a merge as
merging the changes in the parents due to the rebase back into the
original merge. So for a merge M with parents A B C that are rebased to
A' B' C' the rebased merge M' would be constructed by (ignoring shell
quoting issues)

git checkout --detach M
git merge-recursive A -- M A'
tree=$(git write-tree)
git merge-recursive B -- $tree B'
tree=$(git write-tree)
git merge-recursive C -- $tree C'
tree=$(git write-tree)
M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')

This should pull in all the changes from the parents while preserving
any evil conflict resolution in the original merge. It superficially
reminds me of incremental merging [1] but it's so long since I looked at
that I'm not sure if there are any significant similarities.

Best Wishes

Phillip

[1] https://github.com/mhagger/git-imerge

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Igor,

Igor Djordjevic  writes:

> Hi Sergey,
>
> On 01/03/2018 06:39, Sergey Organov wrote:

[...]

>> 
>> Yeah, I now see it myself. I'm sorry for being lazy and not inspecting
>> this more carefully in the first place.
>
> No problem, that`s why we`re discussing it, and I`m glad we`re 
> aligned now, so we can move forward :)
>
>> > So while your original proposal currently seems like it could be 
>> > working nicely for non-interactive rebase (and might be some simpler 
>> > interactive ones), now hitting/acknowledging its first real use 
>> > limit, my additional quick attempt[1] just tries to aid pretty 
>> > interesting case of complicated interactive rebase, too, where we 
>> > might be able to do better as well, still using you original proposal 
>> > as a base idea :)
>> 
>> Yes, thank you for pushing me back to reality! :-) The work and thoughts
>> you are putting into solving the puzzle are greatly appreciated!
>
> You`re welcome, and I am enjoying it :)
>
>> Thinking about it overnight, I now suspect that original proposal had a
>> mistake in the final merge step. I think that what you did is a way to
>> fix it, and I want to try to figure what exactly was wrong in the
>> original proposal and to find simpler way of doing it right.
>> 
>> The likely solution is to use original UM as a merge-base for final
>> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
>> though, as that's exactly UM from which both U1' and U2' have diverged
>> due to rebasing and other history editing.
>
> Yes, this might be it...! ;)
>
> To prove myself it works, I`ve assembled a pretty crazy `-s ours`  
> merge interactive rebase scenario, and it seems this passes the test, 
> ticking all the check boxes (I could think of) :P

I must admit it's quite a relief to hear this. What we now have is so
simple and obvious that it'd be a huge disappointment if didn't work!

> Here, merge commit M is done with `-s ours` (obsoleting branch "B"), 
> plus amended to make it an "evil merge", where a commit B2 from 
> obsoleted branch "B" is cherry picked to "master".

[...]

> There, I hope I didn`t miss any expectation. And, it _seems_ to work 
> exactly as expected :D

That's very nice, to the level of being even suspect! :-)

To avoid falling into euphoria though, we need to keep in mind that
"expectations" is rather vague concept, and we likely still need to stop
for user amendment unless we absolutely sure nothing surprising happens.
I.e., we better require U1'==U2' test to succeed to proceed non-stop
automatically. Besides, it will be somewhat inline with what 'rerere'
does.

> In real life, except for usual possibility for conflicts during 
> commit rebasing, we might experience _three_ possible conflict 
> situations once "rebased" merge itself is to be created - two when 
> rebasing each of temporary merge helper commits, and one on the 
> "rebased" merge itself. This is something where we might think about 
> user experience, not introducing (too much) confusion...

Yeah, this is terribly important issue to take care of! Relative
simplicity of the concept itself raises the chances of finding a
suitable solution, I hope.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 01/03/2018 06:39, Sergey Organov wrote:
> 
> > > (3) ---X1---o---o---o---o---o---X2
> > >|\   |\
> > >| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
> > >| \  |
> > >|  M |
> > >| /  |
> > >\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
> > >
> > 
> > Meh, I hope I`m rushing it now, but for example, if we had decided to 
> > drop commit A2 during an interactive rebase (so losing A2' from 
> > diagram above), wouldn`t U2' still introduce those changes back, once 
> > U1' and U2' are merged, being incorrect/unwanted behavior...? :/
> > 
> > [...]
> 
> Yeah, I now see it myself. I'm sorry for being lazy and not inspecting
> this more carefully in the first place.

No problem, that`s why we`re discussing it, and I`m glad we`re 
aligned now, so we can move forward :)

> > So while your original proposal currently seems like it could be 
> > working nicely for non-interactive rebase (and might be some simpler 
> > interactive ones), now hitting/acknowledging its first real use 
> > limit, my additional quick attempt[1] just tries to aid pretty 
> > interesting case of complicated interactive rebase, too, where we 
> > might be able to do better as well, still using you original proposal 
> > as a base idea :)
> 
> Yes, thank you for pushing me back to reality! :-) The work and thoughts
> you are putting into solving the puzzle are greatly appreciated!

You`re welcome, and I am enjoying it :)

> Thinking about it overnight, I now suspect that original proposal had a
> mistake in the final merge step. I think that what you did is a way to
> fix it, and I want to try to figure what exactly was wrong in the
> original proposal and to find simpler way of doing it right.
> 
> The likely solution is to use original UM as a merge-base for final
> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
> though, as that's exactly UM from which both U1' and U2' have diverged
> due to rebasing and other history editing.

Yes, this might be it...! ;)

To prove myself it works, I`ve assembled a pretty crazy `-s ours`  
merge interactive rebase scenario, and it seems this passes the test, 
ticking all the check boxes (I could think of) :P

Let`s see our starting situation:

 (0) ---X8--B2'--X9 (master)
|\
| A1---A2---A3 (A)
| \
|  M (topic)
| /
\-B1---B2---B3 (B)


Here, merge commit M is done with `-s ours` (obsoleting branch "B"), 
plus amended to make it an "evil merge", where a commit B2 from 
obsoleted branch "B" is cherry picked to "master".

Now, we want to rebase "topic" (M) onto updated "master" (X9), but to 
make things more interesting, we`ll do it interactively, with some 
amendments, drops, additions and even more cherry-picks!

This is what the final result looks like:

 (1) ---X8--B2'--X9 (master)
 |\
 | A12--A2'---B3' (A)
 | \
 |  M' (topic)
 | /
 \-B1'--B3'---B4  (B)


During interactive rebase, on branch "A", we amended A1 into A12, 
dropped A3 and cherry-picked B3. On branch "B", B4 is added, B2' being 
omitted automatically as already present in "master".

So... In comparison to original merge commit M, rebased merge commit 
M' is expected to:

 - Add X9, from updated "master"
 - Have A1 changed to A12, due to A12 commit amendment
 - Keep A2, rebased as A2'
 - Remove A3, due to dropped A3 commit
 - Keep amendment from original (evil) merge commit M
 - Miss B1' like M does B, due to original `-s ours` merge strategy
 - Add B2, cherry-picked as B2' into "master"
 - Add B3, cherry-picked as B3' into "A"
 - Add B4, added to "B"
 - Most important, provide safety mechanism to "fail loud", being 
   aware of non-trivial things going on, allowing to stop for user 
   inspection/decision


There, I hope I didn`t miss any expectation. And, it _seems_ to work 
exactly as expected :D

Not to leave this to imagination only, and hopefully helping others 
to get to speed and possibly discuss this, pointing to still possible 
flaws, I`m adding a demo script[1], showing how this exact example 
works.

Note that script _is_ coined to avoid rebase conflicts, as they`re not 
currently important for the point to be made here.

In real life, except for usual possibility for conflicts during 
commit rebasing, we might experience _three_ possible conflict 
situations once "rebased" merge itself is to be created - two when 
rebasing each of temporary merge helper commits, and one on the 
"rebased" merge itself. This is something where we might think about 
user experience, not introducing (too much) confusion...

Regards, Buga

[1] Demonstration script:
-- >8 --
#!/bin/sh

# rm -rf ./.git
# rm -f ./test.txt

git init

touch ./test.txt
git add -- test.txt

# 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Igor,

Igor Djordjevic  writes:

> Hi Sergey,
>
> On 28/02/2018 06:19, Sergey Organov wrote:
>> 
>> > > (3) ---X1---o---o---o---o---o---X2
>> > >|\   |\
>> > >| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
>> > >| \  |
>> > >|  M |
>> > >| /  |
>> > >\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
>> > >
>> >
>> > Meh, I hope I`m rushing it now, but for example, if we had decided to 
>> > drop commit A2 during an interactive rebase (so losing A2' from 
>> > diagram above), wouldn`t U2' still introduce those changes back, once 
>> > U1' and U2' are merged, being incorrect/unwanted behavior...? :/
>> 
>> I think the method will handle this nicely.
>
> That`s what I thought as well. And then I made a test. And then the 
> test broke... Now, might be the test was flawed in the first place, 
> but thinking about it a bit more, it does seem to make sense not to 
> handle this case nicely :/

Yeah, I now see it myself. I'm sorry for being lazy and not inspecting
this more carefully in the first place.

[...]

> So while your original proposal currently seems like it could be 
> working nicely for non-interactive rebase (and might be some simpler 
> interactive ones), now hitting/acknowledging its first real use 
> limit, my additional quick attempt[1] just tries to aid pretty 
> interesting case of complicated interactive rebase, too, where we 
> might be able to do better as well, still using you original proposal 
> as a base idea :)

Yes, thank you for pushing me back to reality! :-) The work and thoughts
you are putting into solving the puzzle are greatly appreciated!

Thinking about it overnight, I now suspect that original proposal had a
mistake in the final merge step. I think that what you did is a way to
fix it, and I want to try to figure what exactly was wrong in the
original proposal and to find simpler way of doing it right.

The likely solution is to use original UM as a merge-base for final
3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
though, as that's exactly UM from which both U1' and U2' have diverged
due to rebasing and other history editing.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Igor Djordjevic  writes:

[...]
>> > Hmm, still rushing it, but what about adding an additional step, 
>> > something like this:
>> 
>> I think it's unneeded, as it should work fine without it, see another
>> reply.
>
> Unfortunately, I have a broken test case saying different - it could 
> very well be a flawed test, too, but let`s elaborate in that 
> other sub-thread[1], indeed.

Yeah, I was too fast to reply and I was wrong, sorry about it.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Igor,

Igor Djordjevic  writes:

> On 28/02/2018 21:25, Igor Djordjevic wrote:
>> 
>> But U1' and U2' are really to be expected to stay the same in 
>> non-interactive rebase case only...
>
> Just to rephrase to "could be expected" here, meaning still not 
> necessarily in this case, either - I`ve just witnessed 
> non-interactive rebase Johannes previously described[1], merge with 
> `-s ours` (dropping B* patches), plus B1 cherry-picked between 
> X1..X2, eventually coming up with different U1' and U2', too (which 
> would produce a wrong end result, if continued).

Even non-interactive rebase may bring arbitrary asymmetric changes on
both sides of the merge, especially likely when resolving conflicts
needs to take place. OTOH, interactive history editing session could
have merges that we don't intend to edit at all. Overall, neither
non-interactive case is in fact much simpler, nor interactive case is so
much different.

> But I guess this should go to the "complex history" pile, good thing 
> still being that diff safety check between U1' and U2' works as 
> expected, thus automatic merge rebase can be aborted and command 
> given back to the user for closer inspection.

Exactly. This fact hopefully won't stop us from looking for more
suitable automated handling of the general case though. It should still
be our goal to reduce the number of such aborts and to suggest better
merge result for amendment when we are still aborting.

Your proposal hopefully is such a valuable improvement.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

On 28/02/2018 21:25, Igor Djordjevic wrote:
> 
> But U1' and U2' are really to be expected to stay the same in 
> non-interactive rebase case only...

Just to rephrase to "could be expected" here, meaning still not 
necessarily in this case, either - I`ve just witnessed 
non-interactive rebase Johannes previously described[1], merge with 
`-s ours` (dropping B* patches), plus B1 cherry-picked between 
X1..X2, eventually coming up with different U1' and U2', too (which 
would produce a wrong end result, if continued).

But I guess this should go to the "complex history" pile, good thing 
still being that diff safety check between U1' and U2' works as 
expected, thus automatic merge rebase can be aborted and command 
given back to the user for closer inspection.

[1] 
https://public-inbox.org/git/nycvar.qro.7.76.6.1802272330290...@zvavag-6oxh6da.rhebcr.pbec.zvpebfbsg.pbz/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 28/02/2018 07:14, Sergey Organov wrote:
> 
> > > Would additional step as suggested in [1] (using R1 and R2 to "catch" 
> > > interactive rebase additions/amendments/drops, on top of U1' and 
> > > U2'), make more sense (or provide an additional clue, at least)?
> > > 
> > > [1] 
> > > https://public-inbox.org/git/8829c395-fb84-2db0-9288-f7b28fa0d...@gmail.com/
> > 
> > Anyway, from (yet another) ad hoc test, additional step mentioned in 
> > [1] above seems to handle this case, too (merge with `-s ours` 
> > dropping B* patches, plus B1 cherry-picked between X1..X2)
> > 
> > ...
> > 
> > And not just that - it worked with additional interactive rebase 
> > adding, amending and removing commits, on top of all this still 
> > preserving original `-s ours` merge commit evil-merge amendment, too, 
> > all as expected (or so it seems) :P
> 
> Great! I do believe that once we start from some sensible approach, many
> kinds of improvements are possible. I'll definitely need to take close
> look at what you came up with, thanks!
> 
> I'd like to say though that nobody should expect miracles from automated
> rebasing of merges when we get to complex history editing. It will need
> to retreat to manual merge, sooner or later.

I agree, and as I really liked "the feeling" of the original approach 
you described, it felt bad to (presumably) see it failing in what 
doesn`t seem to be neither too complex nor rare situation of dropping 
a commit during an interactive rebase, thus motivation to try to 
improve it, if possible, wasn`t lacking :)
 
Eh, might be I`m just naively ignorant at the moment as well, but I`m 
trying to work my way through it... ;)

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 28/02/2018 06:19, Sergey Organov wrote:
> 
> > > (3) ---X1---o---o---o---o---o---X2
> > >|\   |\
> > >| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
> > >| \  |
> > >|  M |
> > >| /  |
> > >\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
> > >
> >
> > Meh, I hope I`m rushing it now, but for example, if we had decided to 
> > drop commit A2 during an interactive rebase (so losing A2' from 
> > diagram above), wouldn`t U2' still introduce those changes back, once 
> > U1' and U2' are merged, being incorrect/unwanted behavior...? :/
> 
> I think the method will handle this nicely.

That`s what I thought as well. And then I made a test. And then the 
test broke... Now, might be the test was flawed in the first place, 
but thinking about it a bit more, it does seem to make sense not to 
handle this case nicely :/

> When you "drop" A2, will A3' change, or stay intact?

When you "drop" commit A2 from rebasing, patch (diff) A3' does stay 
the same - but resulting tree (snapshot) after applying it does not.

By removing commit A2 from your rebase, you`re saying that changes 
introduced in that commit shouldn`t ever happen in the rebased tree, 
so trees/snapshots of all rebased commits coming after dropped A2 
will have these changes missing, in comparison to trees of original 
commits they`re being rebased from.

> If it changes, say, to A3'', the U1' will change to U1'', and the method
> will propagate the change automatically.
> 
> If it A3' doesn't change, then there are no changes to take.

All true, but note what I wrote in the original message - the issue 
of dropping A2 is not U1, but U2, and that one doesn`t change.

In this case, U1' will correctly represent U1 rebased on top of A3' 
(where A2 is now missing), no problem there.

But on the other end, as U2 holds changes between original merge and 
B3 (being A1, A2, A3 and evil-merge M), it will also still hold 
changes introduced by original A2.

Rebasing it onto B3' brings all these changes along, and once merged 
with U1' to produce "rebased" merge it unexpectedly introduces 
supposedly dropped commit A2 changes in their full glory.

Yes, considering this situation a conflict, as originally proposed, 
by simply noticing that U1' and U2' differ, helps this to fail 
loudly without doing the wrong thing.

But U1' and U2' are really to be expected to stay the same in 
non-interactive rebase case only, where it doesn`t help interactive 
rebase case at all if it is to fail most of the time (where U1' and 
U2' don`t have to be, but should be expected to possibly be different).

So while your original proposal currently seems like it could be 
working nicely for non-interactive rebase (and might be some simpler 
interactive ones), now hitting/acknowledging its first real use 
limit, my additional quick attempt[1] just tries to aid pretty 
interesting case of complicated interactive rebase, too, where we 
might be able to do better as well, still using you original proposal 
as a base idea :)

Regards, Buga

[1] https://public-inbox.org/git/8829c395-fb84-2db0-9288-f7b28fa0d...@gmail.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 28/02/2018 06:44, Sergey Organov wrote:
> 
> > > Here`s our starting position:
> > >
> > > (0) ---X1---o---o---o---o---o---X2 (master)
> > >|\
> > >| A1---A2---A3
> > >| \
> > >|  M (topic)
> > >| /
> > >\-B1---B2---B3
> > >
> > >
> > > Now, we want to rebase merge commit M from X1 onto X2. First, rebase
> > > merge commit parents as usual:
> > >
> > > (1) ---X1---o---o---o---o---o---X2
> > >|\   |\
> > >| A1---A2---A3   | A1'--A2'--A3'
> > >| \  |
> > >|  M |
> > >| /  |
> > >\-B1---B2---B3   \-B1'--B2'--B3'
> > >
> > >
> > > That was commonly understandable part.
> >
> > Good. Let's assume that I want to do this interactively (because let's
> > face it, rebase is boring unless we shake up things a little). And let's
> > assume that A1 is my only change to the README, and that I realized that
> > it was incorrect and I do not want the world to see it, so I drop A1'.
> >
> > Let's see how things go from here:
> >
> > > Now, for "rebasing" the merge commit (keeping possible amendments), we
> > > do some extra work. First, we make two temporary commits on top of old
> > > merge parents, by using exact tree (snapshot) of commit M:
> > >
> > > (2) ---X1---o---o---o---o---o---X2
> > >|\   |\
> > >| A1---A2---A3---U1  | A1'--A2'--A3'
> > >| \  |
> > >|  M |
> > >| /  |
> > >\-B1---B2---B3---U2  \-B1'--B2'--B3'
> >
> > Okay, everything would still be the same except that I still have dropped
> > A1'.
> >
> > > So here, in terms of _snapshots_ (trees, not diffs), U1 = U2 = M.
> > >
> > > Now, we rebase these temporary commits, too:
> > >
> > > (3) ---X1---o---o---o---o---o---X2
> > >|\   |\
> > >| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
> > >| \  |
> > >|  M |
> > >| /  |
> > >\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
> >
> > I still want to drop A1 in this rebase, so A1' is still missing.
> >
> > And now it starts to get interesting.
> >
> > The diff between A3 and U1 does not touch the README, of course, as I said
> > that only A1 changed the README.  But the diff between B3 and U2 does
> > change the README, thanks to M containing A1 change.
> >
> > Therefore, the diff between B3' and U2' will also have this change to the
> > README. That change that I wanted to drop.
> >
> > > As a next step, we merge these temporary commits to produce our
> > > "rebased" merged commit M:
> > >
> > > (4) ---X1---o---o---o---o---o---X2
> > >|\   |\
> > >| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
> > >| \  |  \
> > >|  M |   M'
> > >| /  |  /
> > >\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
> >
> > And here, thanks to B3'..U2' changing the README, M' will also have that
> > change that I wanted to see dropped.
> >
> > Note that A1' is still dropped in my example.
> >
> > > Finally, we drop temporary commits, and record rebased commits A3' 
> > > and B3' as our "rebased" merge commit parents instead (merge commit 
> > > M' keeps its same tree/snapshot state, just gets parents replaced):
> > >
> > > (5) ---X1---o---o---o---o---o---X2
> > >|\   |\
> > >| A1---A2---A3---U1  | A1'--A2'--A3'
> > >| \  | \
> > >|  M |  M'
> > >| /  | /
> > >\-B1---B2---B3---U2  \-B1'--B2'--B3'
> >
> > Now, thanks to U2' being dropped (and A1' *still* being dropped), the
> > change in the README that is still in M' is really only in M'. No other
> > rebased commit has it. That makes it look as if M' introduced this change
> > in addition to the changes that were merged between the merge parents.
> 
> Except that original proposal suggests to cosider this a conflict and
> to stop for amendment, as U1' and U2' now differ.

Thanks for bringing this one up again - I focused on a mere example 
of how the approach could work, not stressing enough the simplicity 
of recognizing when it really doesn`t, which is an important aspect 
to know as well, indeed, supporting much needed trust that no bad 
things could happen behind our back.

Either work as expected, or fail loudly, yes.

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 28/02/2018 06:21, Sergey Organov wrote:
> 
> > > > > (3) ---X1---o---o---o---o---o---X2
> > > > >|\   |\
> > > > >| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
> > > > >| \  |
> > > > >|  M |
> > > > >| /  |
> > > > >\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
> > > > >
> > > >
> > > > Meh, I hope I`m rushing it now, but for example, if we had decided to
> > > > drop commit A2 during an interactive rebase (so losing A2' from
> > > > diagram above), wouldn`t U2' still introduce those changes back, once
> > > > U1' and U2' are merged, being incorrect/unwanted behavior...? :/
> > >
> > > In that case, the method won't work well at all, so I think we need a
> > > different approach.
> > >
> >
> > Hmm, still rushing it, but what about adding an additional step, 
> > something like this:
> 
> I think it's unneeded, as it should work fine without it, see another
> reply.

Unfortunately, I have a broken test case saying different - it could 
very well be a flawed test, too, but let`s elaborate in that 
other sub-thread[1], indeed.

Regards, Buga

[1] https://public-inbox.org/git/87606hoflx@javad.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Igor Djordjevic  writes:

> On 28/02/2018 03:12, Igor Djordjevic wrote:
>> 
>> Would additional step as suggested in [1] (using R1 and R2 to "catch" 
>> interactive rebase additions/amendments/drops, on top of U1' and 
>> U2'), make more sense (or provide an additional clue, at least)?
>> 
>> [1] 
>> https://public-inbox.org/git/8829c395-fb84-2db0-9288-f7b28fa0d...@gmail.com/
>
> Heh, no sleeping tonight :P
>
> Anyway, from (yet another) ad hoc test, additional step mentioned in 
> [1] above seems to handle this case, too (merge with `-s ours` 
> dropping B* patches, plus B1 cherry-picked between X1..X2):
>
> On 28/02/2018 00:27, Johannes Schindelin wrote:
>> 
>> - One of the promises was that the new way would also handle merge
>>   strategies other than recursive. What would happen, for example, if M
>>   was generated using `-s ours` (read: dropping the B* patches' changes)
>>   and if B1 had been cherry-picked into the history between X1..X2?
>> 
>>   Reverting R would obviously revert those B1 changes, even if B1' would
>>   obviously not even be part of the rebased history!
>> 
>> Yes, I agree that this `-s ours` example is quite concocted, but the point
>> of this example is not how plausible it is, but how easy it is to come up
>> with a scenario where this design to "rebase merge commits" results in
>> very, very unwanted behavior.
>
> And not just that - it worked with additional interactive rebase 
> adding, amending and removing commits, on top of all this still 
> preserving original `-s ours` merge commit evil-merge amendment, too, 
> all as expected (or so it seems) :P

Great! I do believe that once we start from some sensible approach, many
kinds of improvements are possible. I'll definitely need to take close
look at what you came up with, thanks!

I'd like to say though that nobody should expect miracles from automated
rebasing of merges when we get to complex history editing. It will need
to retreat to manual merge, sooner or later.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Jacob Keller  writes:

> On Tue, Feb 27, 2018 at 10:14 AM, Junio C Hamano  wrote:
>> Sergey Organov  writes:
>>
>>> You've already bit this poor thingy to death. Please rather try your
>>> teeth on the proposed Trivial Merge (TM) method.
>>
>> Whatever you do, do *NOT* call any part of your proposal "trivial
>> merge", unless you are actually using the term to mean what Git
>> calls "trivial merge".  The phrase has an established meaning in Git
>> and your attempt to abuse it to mean something entirely different is
>> adding unnecessary hindrance for other people to understand what you
>> want to perform.
>
> Agreed, I think we need better terminology here, the current words for
> (TM) are definitely *not* trivial merges. Same for "angel merge", I
> don't think that term really works well either.

Agreed.

How do we call a merge that introduces no differences on either side of
the merge then? Is there some English for even more trivial than what
Git calls "trivial merge"?

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin  writes:
> Hi Buga,
>
> thank you for making this a lot more understandable to this thick
> developer.
>
> On Tue, 27 Feb 2018, Igor Djordjevic wrote:
>
>> On 27/02/2018 19:55, Igor Djordjevic wrote:
>> > 
>> > It would be more along the lines of "(1) rebase old merge commit parents, 
>> > (2) generate separate diff between old merge commit and each of its 
>> > parents, (3) apply each diff to their corresponding newly rebased 
>> > parent respectively (as a temporary commit, one per rebased parent), 
>> > (4) merge these temporary commits to generate 'rebased' merge commit, 
>> > (5) drop temporary commits, recording their parents as parents of 
>> > 'rebased' merge commit (instead of dropped temporary commits)".
>> > 
>> > Implementation wise, steps (2) and (3) could also be done by simply 
>> > copying old merge commit _snapshot_ on top of each of its parents as 
>> > a temporary, non-merge commit, then rebasing (cherry-picking) these 
>> > temporary commits on top of their rebased parent commits to produce 
>> > rebased temporary commits (to be merged for generating 'rebased' 
>> > merge commit in step (4)).
>> 
>> For those still tagging along (and still confused), here are some 
>> diagrams (following what Sergey originally described). Note that 
>> actual implementation might be even simpler, but I believe it`s a bit 
>> easier to understand like this, using some "temporary" commits approach.
>> 
>> Here`s our starting position:
>> 
>> (0) ---X1---o---o---o---o---o---X2 (master)
>>|\
>>| A1---A2---A3
>>| \
>>|  M (topic)
>>| /
>>\-B1---B2---B3
>> 
>> 
>> Now, we want to rebase merge commit M from X1 onto X2. First, rebase
>> merge commit parents as usual:
>> 
>> (1) ---X1---o---o---o---o---o---X2
>>|\   |\
>>| A1---A2---A3   | A1'--A2'--A3'
>>| \  |
>>|  M |
>>| /  |
>>\-B1---B2---B3   \-B1'--B2'--B3'
>> 
>> 
>> That was commonly understandable part.
>
> Good. Let's assume that I want to do this interactively (because let's
> face it, rebase is boring unless we shake up things a little). And let's
> assume that A1 is my only change to the README, and that I realized that
> it was incorrect and I do not want the world to see it, so I drop A1'.
>
> Let's see how things go from here:
>
>> Now, for "rebasing" the merge commit (keeping possible amendments), we
>> do some extra work. First, we make two temporary commits on top of old
>> merge parents, by using exact tree (snapshot) of commit M:
>> 
>> (2) ---X1---o---o---o---o---o---X2
>>|\   |\
>>| A1---A2---A3---U1  | A1'--A2'--A3'
>>| \  |
>>|  M |
>>| /  |
>>\-B1---B2---B3---U2  \-B1'--B2'--B3'
>
> Okay, everything would still be the same except that I still have dropped
> A1'.
>
>> So here, in terms of _snapshots_ (trees, not diffs), U1 = U2 = M.
>> 
>> Now, we rebase these temporary commits, too:
>> 
>> (3) ---X1---o---o---o---o---o---X2
>>|\   |\
>>| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
>>| \  |
>>|  M |
>>| /  |
>>\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
>
> I still want to drop A1 in this rebase, so A1' is still missing.
>
> And now it starts to get interesting.
>
> The diff between A3 and U1 does not touch the README, of course, as I said
> that only A1 changed the README.  But the diff between B3 and U2 does
> change the README, thanks to M containing A1 change.
>
> Therefore, the diff between B3' and U2' will also have this change to the
> README. That change that I wanted to drop.
>
>> As a next step, we merge these temporary commits to produce our
>> "rebased" merged commit M:
>> 
>> (4) ---X1---o---o---o---o---o---X2
>>|\   |\
>>| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
>>| \  |  \
>>|  M |   M'
>>| /  |  /
>>\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
>
> And here, thanks to B3'..U2' changing the README, M' will also have that
> change that I wanted to see dropped.
>
> Note that A1' is still dropped in my example.
>
>> Finally, we drop temporary commits, and record rebased commits A3' 
>> and B3' as our "rebased" merge commit parents instead (merge commit 
>> M' keeps its same tree/snapshot state, just gets parents replaced):
>> 
>> (5) ---X1---o---o---o---o---o---X2
>>|\   |\
>>| A1---A2---A3---U1  | A1'--A2'--A3'
>>| \  | \

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Junio C Hamano  writes:

> Igor Djordjevic  writes:
>
>> On 27/02/2018 20:59, Igor Djordjevic wrote:
>>> 
>>> (3) ---X1---o---o---o---o---o---X2
>>>|\   |\
>>>| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
>>>| \  |
>>>|  M |
>>>| /  |
>>>\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
>>> 
>>
>> Meh, I hope I`m rushing it now, but for example, if we had decided to 
>> drop commit A2 during an interactive rebase (so losing A2' from 
>> diagram above), wouldn`t U2' still introduce those changes back, once 
>> U1' and U2' are merged, being incorrect/unwanted behavior...? :/
>>
>> p.s. Looks like Johannes already elaborated on this in the meantime, 
>> let`s see... (goes reading that other e-mail[1])
>
> As long as we are talking about rebase that allows us users to
> adjust and make changes ("rebase -i" being the prime and most
> flexible example), it is easy to imagine that A1'--A3' and B1'--B3'
> have almost no relation to their original counterparts.  After all,
> mechanical merge won't be able to guess the intention of the change
> humans make, so depending on what happend during X1 and X2 that
> happend outside of these two topics, what's required to bring series
> A and B to series A' and B' can be unlimited.

You discuss some different method here. In my original proposal U1' and
U2' are to be merged. Nothing should be replayed on top of them. I.e.,
U1' is _already_ the result of replaying difference between M and A3 on
top of A3'.

> So from that alone, it should be clear that replaying difference
> between M and A3 (and M and B3) on top of U1' and U2' is hopeless as a
> general solution.

Getting U1' from U1 is the same complexity as getting A3' from A3, with
the same caveats. So, as general solution, it's as good as rebase of
non-merge commit.

> It is acceptable as long as a solution fails loudly when it does the
> wrong thing, but I do not think the apporach can produce incorrect
> result silently, as your example shows above.

I still believe the method handles simple cases automatically and
correctly and allows to immediately stop for amendment should something
suspect appear.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Igor Djordjevic  writes:

> On 28/02/2018 01:36, Jacob Keller wrote:
>> 
>> > > (3) ---X1---o---o---o---o---o---X2
>> > >|\   |\
>> > >| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
>> > >| \  |
>> > >|  M |
>> > >| /  |
>> > >\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
>> > >
>> >
>> > Meh, I hope I`m rushing it now, but for example, if we had decided to
>> > drop commit A2 during an interactive rebase (so losing A2' from
>> > diagram above), wouldn`t U2' still introduce those changes back, once
>> > U1' and U2' are merged, being incorrect/unwanted behavior...? :/
>> 
>> In that case, the method won't work well at all, so I think we need a
>> different approach.
>> 
>
> Hmm, still rushing it, but what about adding an additional step, 
> something like this:

I think it's unneeded, as it should work fine without it, see another
reply.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Igor Djordjevic  writes:

> On 27/02/2018 20:59, Igor Djordjevic wrote:
>> 
>> (3) ---X1---o---o---o---o---o---X2
>>|\   |\
>>| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
>>| \  |
>>|  M |
>>| /  |
>>\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
>> 
>
> Meh, I hope I`m rushing it now, but for example, if we had decided to 
> drop commit A2 during an interactive rebase (so losing A2' from 
> diagram above), wouldn`t U2' still introduce those changes back, once 
> U1' and U2' are merged, being incorrect/unwanted behavior...? :/

I think the method will handle this nicely.

When you "drop" A2, will A3' change, or stay intact?

If it changes, say, to A3'', the U1' will change to U1'', and the method
will propagate the change automatically.

If it A3' doesn't change, then there are no changes to take.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Junio C Hamano  writes:

> Sergey Organov  writes:
>
>> You've already bit this poor thingy to death. Please rather try your
>> teeth on the proposed Trivial Merge (TM) method.
>
> Whatever you do, do *NOT* call any part of your proposal "trivial
> merge", unless you are actually using the term to mean what Git
> calls "trivial merge".  The phrase has an established meaning in Git
> and your attempt to abuse it to mean something entirely different is
> adding unnecessary hindrance for other people to understand what you
> want to perform.

Yeah, got it. It's confusing indeed.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

On 28/02/2018 03:12, Igor Djordjevic wrote:
> 
> Would additional step as suggested in [1] (using R1 and R2 to "catch" 
> interactive rebase additions/amendments/drops, on top of U1' and 
> U2'), make more sense (or provide an additional clue, at least)?
> 
> [1] 
> https://public-inbox.org/git/8829c395-fb84-2db0-9288-f7b28fa0d...@gmail.com/

Heh, no sleeping tonight :P

Anyway, from (yet another) ad hoc test, additional step mentioned in 
[1] above seems to handle this case, too (merge with `-s ours` 
dropping B* patches, plus B1 cherry-picked between X1..X2):

On 28/02/2018 00:27, Johannes Schindelin wrote:
> 
> - One of the promises was that the new way would also handle merge
>   strategies other than recursive. What would happen, for example, if M
>   was generated using `-s ours` (read: dropping the B* patches' changes)
>   and if B1 had been cherry-picked into the history between X1..X2?
> 
>   Reverting R would obviously revert those B1 changes, even if B1' would
>   obviously not even be part of the rebased history!
> 
> Yes, I agree that this `-s ours` example is quite concocted, but the point
> of this example is not how plausible it is, but how easy it is to come up
> with a scenario where this design to "rebase merge commits" results in
> very, very unwanted behavior.

And not just that - it worked with additional interactive rebase 
adding, amending and removing commits, on top of all this still 
preserving original `-s ours` merge commit evil-merge amendment, too, 
all as expected (or so it seems) :P

Again, for the brave ones, here`s another messy test script (not 
tightly related to the last discussed diagrams, but based on my 
original "demonstration" script[2])... Hopefully I get to make a 
proper (and sane) sample soon... if not missing something here in the 
first place ;)

Regards, Buga

[2] https://public-inbox.org/git/bbe64321-4d3a-d3fe-8bb9-58b600fab...@gmail.com/

-- 8< --
#!/bin/sh

# rm -rf ./.git
# rm -f ./test.txt

git init

touch ./test.txt
git add -- test.txt

for i in {1..20}
do
echo A$i >>test.txt
git commit -am "A$i"
done

git checkout -b b1
sed -i '3iB11' test.txt
git commit -am "B11"
sed -i '7iB12' test.txt
git commit -am "B12"

git checkout -b b2 HEAD^
sed -i '16iB21' test.txt
git commit -am "B21"
sed -i '18iB22' test.txt
git commit -am "B22"

git checkout -b merge b1
git merge -s ours --no-commit b2
sed -i '12iX' test.txt # amend merge commit
git commit -am "M"
git tag original-merge

git checkout master
git cherry-pick b2
for i in {1..5}
do
j=`expr "$i" + 20`
sed -i "${i}iA${j}" test.txt
git commit -am "A$j"
done

# simple/naive demonstration of proposed merge rebasing logic
# using described "Trivial Merge" (TM, or "Angel Merge"),
# preserving merge commit manual amendments, but still respecting
# interactively rebased added/modified/dropped commits :)

# read -p "Press enter to continue"
git checkout b1
git cherry-pick -m1 original-merge && git tag U1
git reset --hard HEAD^^ # drop U1 and last b1 commit
sed -i '/B11/c\B' test.txt
git commit -a --amend --no-edit
git rebase master
git cherry-pick U1 && git tag U1-prime

# read -p "Press enter to continue"
git checkout b2
git cherry-pick -m2 original-merge && git tag U2
git reset --hard HEAD^ # drop U2
git rebase master
sed -i '20iBX' test.txt
git commit -am "BX" # add new commit
git cherry-pick U2 && git tag U2-prime

git diff U1 U1-prime | git apply --3way && git commit -m "U2-second" && git tag 
U2-second
git checkout b1
git diff U2 U2-prime | git apply --3way && git commit -m "U1-second" && git tag 
U1-second

# read -p "Press enter to continue"
git branch -f merge b1
git checkout merge
git merge b2 --no-commit
git commit -a --reuse-message original-merge
git tag angel-merge

# read -p "Press enter to continue"
git reset --hard b1^
git read-tree --reset angel-merge
git update-ref refs/heads/merge "$(git show -s --format=%B original-merge | git 
commit-tree "$(git write-tree)" -p "$(git rev-parse b1^^)" -p "$(git rev-parse 
b2^^)")"
git tag -f angel-merge
git checkout angel-merge .
git branch -f b1 b1^^
git branch -f b2 b2^^

# show resulting graph
echo
git log --all --decorate --oneline --graph

# comparison between original merge and rebased merge,
# showing merge commit amendment "X" being preserved during rebase
# (not shown in diff)
echo
echo 'diff original-merge angel-merge:'
git diff original-merge angel-merge

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Junio,

On 28/02/2018 01:10, Junio C Hamano wrote:
> 
> > > (3) ---X1---o---o---o---o---o---X2
> > >|\   |\
> > >| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
> > >| \  |
> > >|  M |
> > >| /  |
> > >\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
> > >
> >
> > Meh, I hope I`m rushing it now, but for example, if we had decided to 
> > drop commit A2 during an interactive rebase (so losing A2' from 
> > diagram above), wouldn`t U2' still introduce those changes back, once 
> > U1' and U2' are merged, being incorrect/unwanted behavior...? :/
> 
> As long as we are talking about rebase that allows us users to
> adjust and make changes ("rebase -i" being the prime and most
> flexible example), it is easy to imagine that A1'--A3' and B1'--B3'
> have almost no relation to their original counterparts.  After all,
> mechanical merge won't be able to guess the intention of the change
> humans make, so depending on what happend during X1 and X2 that
> happend outside of these two topics, what's required to bring series
> A and B to series A' and B' can be unlimited.  So from that alone,
> it should be clear that replaying difference between M and A3 (and M
> and B3) on top of U1' and U2' is hopeless as a general solution.

Yeah, I`ve encountered it in my (trivial) test case :(

> It is acceptable as long as a solution fails loudly when it does the
> wrong thing, but I do not think the apporach can produce incorrect
> result silently, as your example shows above.

Hmm, I think my example doesn`t even try to prevent failing, but it 
should otherwise be perfectly capable of doing so (and doing it 
loudly) - for example, it`s enough to diff U1' and U2' - if not the 
same, user might want to confirm the "rebased" merge outcome, as 
either something went wrong, or interactive rebase happened... or 
both :) (it`s what Sergey originally explained, seeming to be a solid 
safety net, though more testing would be good)

> What you _COULD_ learn from an old merge is to compute:
> 
> - a trial and mechanical merge between A3 and B3; call that pre-M
> 
> - diff to bring pre-M to M (call that patch evil-M); which is
>   what the person who made M did to resolve the textual and
>   semantic conflicts necessary to merge these two topics.
> 
> Then when merging A3' and B3', you could try to mechanically merge
> them (call that pre-M'), and apply evil-M, which may apply cleanly
> on top of pre-M', or it may not.  When there aren't so huge a
> difference between series A and A' (and series B and B'), the result
> would probably be a moral equivalent of Sergay's "replay" (so this
> approach will also silently produce a wrong result without human
> supervision).  One edge the evil-M approach has over Sergey's "dual
> cherry pick" is that it separates and highlights non-mechanical
> conflict resolution out of mechanical merges in a human readable
> form (i.e. the patch evil-M).

This seems to be what Johannes wrote about[1], too, but also 
something I think would be good to avoid, if possible, not 
complicating it too much :P

Maybe something like that latest thought[2] could help, using 
additional R1 and R2 commits to handle interactive rebase 
additions/amendments/drops, on top of U1' and U2'? Yeah, and 
that`s not complicating it... ;) :D

Regards, Buga

[1] 
https://public-inbox.org/git/nycvar.qro.7.76.6.1802272330290...@zvavag-6oxh6da.rhebcr.pbec.zvpebfbsg.pbz/
[2] https://public-inbox.org/git/8829c395-fb84-2db0-9288-f7b28fa0d...@gmail.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Johannes,

On 28/02/2018 00:27, Johannes Schindelin wrote:
> 
> thank you for making this a lot more understandable to this thick
> developer.

Hehe, no problem, it primarily served fighting my own thickness ;)

> > Finally, we drop temporary commits, and record rebased commits A3' 
> > and B3' as our "rebased" merge commit parents instead (merge commit 
> > M' keeps its same tree/snapshot state, just gets parents replaced):
> >
> > (5) ---X1---o---o---o---o---o---X2
> >|\   |\
> >| A1---A2---A3---U1  | A1'--A2'--A3'
> >| \  | \
> >|  M |  M'
> >| /  | /
> >\-B1---B2---B3---U2  \-B1'--B2'--B3'
> 
> ...
> 
> In my example, where I dropped A1' specifically so that that embarrasingly
> incorrect change to the README would not be seen by the world, though, the
> evil merge would be truly evil: it would show said change to the world.
> The exact opposite of what I wanted.

Yeah, I`m afraid that`s what my testing produced as well :( Back to 
the drawing board...

> It would have been nice to have such a simple solution ;-)

Eh, the worst thing is the feeling I have, like it`s just around the 
corner, but we`re somehow missing it :P

> So the most obvious way to try to fix this design would be to recreate the
> original merge first, even with merge conflicts, and then trying to use the
> diff between that and the actual original merge commit.

For simplicity sake, this is something I would like to avoid (if 
possible), and also for the reasons you mentioned yourself:

> Now, would this work?
> 
> I doubt it, for at least two reasons:
> 
> - if there are merge conflicts between A3/B3 and between A3'/B3', those
>   merge conflicts will very likely look very different, and the conflicts
>   when reverting R will contain those nested conflicts: utterly confusing.
>   And those conflicts will look even more confusing if a patch (such as
>   A1') was dropped during an interactive rebase.
> 
> - One of the promises was that the new way would also handle merge
>   strategies other than recursive. What would happen, for example, if M
>   was generated using `-s ours` (read: dropping the B* patches' changes)
>   and if B1 had been cherry-picked into the history between X1..X2?
> 
>   Reverting R would obviously revert those B1 changes, even if B1' would
>   obviously not even be part of the rebased history!
> 
> ...
> 
> But maybe I missed something obvious, and the design can still be fixed
> somehow?

Would additional step as suggested in [1] (using R1 and R2 to "catch" 
interactive rebase additions/amendments/drops, on top of U1' and 
U2'), make more sense (or provide an additional clue, at least)?

It`s late here, and I`m really rushing it now, so please forgive me if 
it`s a stupid one... :$

Regards, Buga

[1] https://public-inbox.org/git/8829c395-fb84-2db0-9288-f7b28fa0d...@gmail.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

On 28/02/2018 02:33, Igor Djordjevic wrote:
> 
> This seems to be working inside my (too trivial?) test case, for 
> interactive adding, dropping, and amending of rebased commits, 
> resulting "rebased" merge containing all the added/modified/dropped 
> changes, plus the original merge amendment, all as expected :P

In case anyone has a wish to examine my (now pretty messy) test 
script, here it is - sorry for not having time to clean it up! :(

What I get when I diff original and "rebased" merge is this:

diff --git a/test.txt b/test.txt
index a82470b..d458032 100644
--- a/test.txt
+++ b/test.txt
@@ -1,10 +1,14 @@
+A21
+A22
+A23
+A24
+A25
 A1
 A2
-B11
+B
 A3
 A4
 A5
-B12
 A6
 A7
 A8
@@ -14,6 +18,7 @@ A10
 A11
 A12
 A13
+BX
 A14
 B2
 A15


... where A21 to A25 are additions due to new base, B11 was 
interactively amended to B, B12 was interactively dropped, and BX 
interactively added :)

We don`t see line X here, being an "evil merge" amendment being 
correctly preserved from original merge commit (thus not a 
difference). If we do `git show` of the "rebased" merge, we get this, 
as expected:

diff --cc test.txt
index b173cef,fad39a8..d458032
--- a/test.txt
+++ b/test.txt
@@@ -13,6 -13,6 +13,7 @@@ A
  A7
  A8
  A9
++X
  A10
  A11
  A12


Regards, Buga

-- 8< --
#!/bin/sh

# rm -rf ./.git
# rm -f ./test.txt

git init

touch ./test.txt
git add -- test.txt

for i in {1..20}
do
echo A$i >>test.txt
git commit -am "A$i"
done

git checkout -b b1
sed -i '3iB11' test.txt
git commit -am "B11"
sed -i '7iB12' test.txt
git commit -am "B12"

git checkout -b b2 HEAD^
sed -i '16iB2' test.txt
git commit -am "B2"

git checkout -b merge b1
git merge --no-commit b2
sed -i '12iX' test.txt # amend merge commit
git commit -am "M"
git tag original-merge

git checkout master
for i in {1..5}
do
j=`expr "$i" + 20`
sed -i "${i}iA${j}" test.txt
git commit -am "A$j"
done

# simple/naive demonstration of proposed merge rebasing logic
# using described "Trivial Merge" (TM, or "Angel Merge"),
# preserving merge commit manual amendments, but still respecting
# interactively rebased added/modified/dropped commits :)

# read -p "Press enter to continue"
git checkout b1
git cherry-pick -m1 original-merge && git tag U1
git reset --hard HEAD^^ # drop U1 and last b1 commit
sed -i '/B11/c\B' test.txt
git commit -a --amend --no-edit
git rebase master
git cherry-pick U1 && git tag U1-prime

# read -p "Press enter to continue"
git checkout b2
git cherry-pick -m2 original-merge && git tag U2
git reset --hard HEAD^ # drop U2
git rebase master
sed -i '20iBX' test.txt
git commit -am "BX" # add new commit
git cherry-pick U2 && git tag U2-prime

git diff U1 U1-prime | git apply --3way && git commit -m "U2-second" && git tag 
U2-second
git checkout b1
git diff U2 U2-prime | git apply --3way && git commit -m "U1-second" && git tag 
U1-second

# read -p "Press enter to continue"
git branch -f merge b1
git checkout merge
git merge b2 --no-commit
git commit -a --reuse-message original-merge
git tag angel-merge

# read -p "Press enter to continue"
git reset --hard b1^
git read-tree --reset angel-merge
git update-ref refs/heads/merge "$(git show -s --format=%B original-merge | git 
commit-tree "$(git write-tree)" -p "$(git rev-parse b1^^)" -p "$(git rev-parse 
b2^^)")"
git tag -f angel-merge
git checkout angel-merge .
git branch -f b1 b1^^
git branch -f b2 b2^^

# show resulting graph
echo
git log --all --decorate --oneline --graph

# comparison between original merge and rebased merge,
# showing merge commit amendment "X" being preserved during rebase
# (not shown in diff)
echo
echo 'diff original-merge angel-merge:'
git diff original-merge angel-merge

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

On 28/02/2018 01:36, Jacob Keller wrote:
> 
> > > (3) ---X1---o---o---o---o---o---X2
> > >|\   |\
> > >| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
> > >| \  |
> > >|  M |
> > >| /  |
> > >\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
> > >
> >
> > Meh, I hope I`m rushing it now, but for example, if we had decided to
> > drop commit A2 during an interactive rebase (so losing A2' from
> > diagram above), wouldn`t U2' still introduce those changes back, once
> > U1' and U2' are merged, being incorrect/unwanted behavior...? :/
> 
> In that case, the method won't work well at all, so I think we need a
> different approach.
> 

Hmm, still rushing it, but what about adding an additional step, 
something like this: 
 
 (4) ---X1---o---o---o---o---o---X2
|\   |\
| A1---A2---A3---U1  | A1'--A2'--A3'--U1'--R1
| \  |
|  M |
| /  |
\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'--R2


... where:

  R1 = git diff U2 U2' | git apply --3way
  R2 = git diff U1 U1' | git apply --3way

(this is just to explain the point, might be there is a better way to 
produce Rx)

So, we still use Ux' to preserve merge commit M amendments, but also 
Rx to catch any changes happening between Ux and Ux' caused by 
interactive rebase commit manipulation (add/amend/drop).

Note that R*1* is produced by applying diff from U*2*' side, and vice 
versa (as it`s the other side that can erroneously introduce dropped 
commit changes back, like U2' in case of dropped A2').

>From here we continue as before - merging R1 and R2, then rewriting 
merge commit parents to point to A3' and B3' (dropping Ux` and Rx).

This seems to be working inside my (too trivial?) test case, for 
interactive adding, dropping, and amending of rebased commits, 
resulting "rebased" merge containing all the added/modified/dropped 
changes, plus the original merge amendment, all as expected :P

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Jacob Keller]

On Tue, Feb 27, 2018 at 3:40 PM, Igor Djordjevic
 wrote:
> On 27/02/2018 20:59, Igor Djordjevic wrote:
>>
>> (3) ---X1---o---o---o---o---o---X2
>>|\   |\
>>| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
>>| \  |
>>|  M |
>>| /  |
>>\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
>>
>
> Meh, I hope I`m rushing it now, but for example, if we had decided to
> drop commit A2 during an interactive rebase (so losing A2' from
> diagram above), wouldn`t U2' still introduce those changes back, once
> U1' and U2' are merged, being incorrect/unwanted behavior...? :/
>
> p.s. Looks like Johannes already elaborated on this in the meantime,
> let`s see... (goes reading that other e-mail[1])
>
> [1] 
> https://public-inbox.org/git/nycvar.qro.7.76.6.1802272330290...@zvavag-6oxh6da.rhebcr.pbec.zvpebfbsg.pbz/


In that case, the method won't work well at all, so I think we need a
different approach.

Thanks,
Jake

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Jacob Keller]

On Tue, Feb 27, 2018 at 10:14 AM, Junio C Hamano  wrote:
> Sergey Organov  writes:
>
>> You've already bit this poor thingy to death. Please rather try your
>> teeth on the proposed Trivial Merge (TM) method.
>
> Whatever you do, do *NOT* call any part of your proposal "trivial
> merge", unless you are actually using the term to mean what Git
> calls "trivial merge".  The phrase has an established meaning in Git
> and your attempt to abuse it to mean something entirely different is
> adding unnecessary hindrance for other people to understand what you
> want to perform.

Agreed, I think we need better terminology here, the current words for
(TM) are definitely *not* trivial merges. Same for "angel merge", I
don't think that term really works well either.

The goal of the process is to split the merge apart to its components
for each side branch and then bring them back together after applying
them to the newly rebased branches.

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Jacob Keller]

On Tue, Feb 27, 2018 at 8:21 AM, Johannes Schindelin
 wrote:
> Hi Jake,
>
> On Mon, 26 Feb 2018, Jacob Keller wrote:
>
>> On Mon, Feb 26, 2018 at 4:07 PM, Johannes Schindelin
>>  wrote:
>> >
>> > On Tue, 20 Feb 2018, Igor Djordjevic wrote:
>> >
>> >> I`m really interested in this topic, which seems to (try to) address the
>> >> only "bad feeling" I had with rebasing merges - being afraid of silently
>> >> losing amendments by actually trying to "replay" the merge (where
>> >> additional and possibly important context is missing), instead of really
>> >> "rebasing" it (somehow).
>> >
>> > If those amendments are what you are worried about, why not address them
>> > specifically?
>> >
>> > In other words, rather than performing the equivalent of
>> >
>> > git show ^! | git apply
>> >
>> > (which would of course completely ignore if the rewritten ^2
>> > dropped a patch, amended a patch, or even added a new patch), what you
>> > really want is to figure out what changes the user made when merging, and
>> > what merge strategy was used to begin with.
>> >
>> > To see what I mean, look at the output of `git show 0fd90daba8`: it shows
>> > how conflicts were resolved. By necessity, this is more complicated than a
>> > simple diff: it is *not* as simple as taking a diff between two revisions
>> > and applying that diff to a third revision. There were (at least) three
>> > revisions involved in the original merge commit, and recreating that merge
>> > commit faithfully means to represent the essence of the merge commit
>> > faithfully enough to be able to replay it on a new set of at least three
>> > revisions.  That can be simplified to two-way diffs only in very, very
>> > special circumstances, and in all other cases this simplification will
>> > simply fall on its nose.
>> >
>> > If the proposed solution was to extend `git apply` to process combined
>> > diffs, I would agree that we're on to something. That is not the proposed
>> > solution, though.
>> >
>> > In short: while I am sympathetic to the desire to keep things simple,
>> > the idea to somehow side-step replaying the original merge seems to be
>> > *prone* to be flawed. Any system that cannot accommodate
>> > dropped/changed/added commits on either side of a merge is likely to be
>> > too limited to be useful.
>> >
>>
>>
>> The reason Sergey's solution works is because he cherry picks the
>> merge using each parent first, and then merges the result of those. So
>> each branch of the merge gets one, and then you merge the result of
>> those cherry-picks. This preservers amendments and changes properly,
>> and should result in a good solution.
>
> I saw your patch trying to add a minimal example, and I really want to run
> away screaming.
>
> Do you have any way to describe the idea in a simple, 3-5 lines long
> paragraph?
>
> So far, I just know that it is some sort of confusing criss-cross
> cherry-picking and merging and stuff, but nothing in those steps shouts
> out to me what the *idea* is.
>

Sergey's posted explained it more in detail, at
https://public-inbox.org/git/87y3jtqdyg@javad.com/

I was mostly just attempting to re-create it in a test case to show
that it could work.

> If it would be something like "recreate the old merge, with merge
> conflicts and all, then generate the diff to the actual tree of the merge
> commit, then apply that to the newly-generated merge", I would understand.
>

It's more or less:

Rebase each parent, then cherry-pick -m the original merge to that
parent, then you merge the result of each cherry-pick, then use the
resulting final merged tree to create the merge pointing at the real
parents instead of the cherry-pick merges.

> I would still suspect that -s ours would be a hard nut for that method,
> but I would understand that idea.
>

The goal of the process isn't to know or understand the "-s ours"
strategy, but simply re-create the contents of the original merge
faithfully, while still preserving the changes done when rebasing the
side branches. Thus it should re-create the contents generated by "-s
ours" the first time, but it doesn't need to do or know anything
special about how the content was created.

> Thanks,
> Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Junio C Hamano]

Igor Djordjevic  writes:

> On 27/02/2018 20:59, Igor Djordjevic wrote:
>> 
>> (3) ---X1---o---o---o---o---o---X2
>>|\   |\
>>| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
>>| \  |
>>|  M |
>>| /  |
>>\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
>> 
>
> Meh, I hope I`m rushing it now, but for example, if we had decided to 
> drop commit A2 during an interactive rebase (so losing A2' from 
> diagram above), wouldn`t U2' still introduce those changes back, once 
> U1' and U2' are merged, being incorrect/unwanted behavior...? :/
>
> p.s. Looks like Johannes already elaborated on this in the meantime, 
> let`s see... (goes reading that other e-mail[1])

As long as we are talking about rebase that allows us users to
adjust and make changes ("rebase -i" being the prime and most
flexible example), it is easy to imagine that A1'--A3' and B1'--B3'
have almost no relation to their original counterparts.  After all,
mechanical merge won't be able to guess the intention of the change
humans make, so depending on what happend during X1 and X2 that
happend outside of these two topics, what's required to bring series
A and B to series A' and B' can be unlimited.  So from that alone,
it should be clear that replaying difference between M and A3 (and M
and B3) on top of U1' and U2' is hopeless as a general solution.

It is acceptable as long as a solution fails loudly when it does the
wrong thing, but I do not think the apporach can produce incorrect
result silently, as your example shows above.

What you _COULD_ learn from an old merge is to compute:

- a trial and mechanical merge between A3 and B3; call that pre-M

- diff to bring pre-M to M (call that patch evil-M); which is
  what the person who made M did to resolve the textual and
  semantic conflicts necessary to merge these two topics.

Then when merging A3' and B3', you could try to mechanically merge
them (call that pre-M'), and apply evil-M, which may apply cleanly
on top of pre-M', or it may not.  When there aren't so huge a
difference between series A and A' (and series B and B'), the result
would probably be a moral equivalent of Sergay's "replay" (so this
approach will also silently produce a wrong result without human
supervision).  One edge the evil-M approach has over Sergey's "dual
cherry pick" is that it separates and highlights non-mechanical
conflict resolution out of mechanical merges in a human readable
form (i.e. the patch evil-M).

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

On 27/02/2018 20:59, Igor Djordjevic wrote:
> 
> (3) ---X1---o---o---o---o---o---X2
>|\   |\
>| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
>| \  |
>|  M |
>| /  |
>\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'
> 

Meh, I hope I`m rushing it now, but for example, if we had decided to 
drop commit A2 during an interactive rebase (so losing A2' from 
diagram above), wouldn`t U2' still introduce those changes back, once 
U1' and U2' are merged, being incorrect/unwanted behavior...? :/

p.s. Looks like Johannes already elaborated on this in the meantime, 
let`s see... (goes reading that other e-mail[1])

[1] 
https://public-inbox.org/git/nycvar.qro.7.76.6.1802272330290...@zvavag-6oxh6da.rhebcr.pbec.zvpebfbsg.pbz/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Buga,

thank you for making this a lot more understandable to this thick
developer.

On Tue, 27 Feb 2018, Igor Djordjevic wrote:

> On 27/02/2018 19:55, Igor Djordjevic wrote:
> > 
> > It would be more along the lines of "(1) rebase old merge commit parents, 
> > (2) generate separate diff between old merge commit and each of its 
> > parents, (3) apply each diff to their corresponding newly rebased 
> > parent respectively (as a temporary commit, one per rebased parent), 
> > (4) merge these temporary commits to generate 'rebased' merge commit, 
> > (5) drop temporary commits, recording their parents as parents of 
> > 'rebased' merge commit (instead of dropped temporary commits)".
> > 
> > Implementation wise, steps (2) and (3) could also be done by simply 
> > copying old merge commit _snapshot_ on top of each of its parents as 
> > a temporary, non-merge commit, then rebasing (cherry-picking) these 
> > temporary commits on top of their rebased parent commits to produce 
> > rebased temporary commits (to be merged for generating 'rebased' 
> > merge commit in step (4)).
> 
> For those still tagging along (and still confused), here are some 
> diagrams (following what Sergey originally described). Note that 
> actual implementation might be even simpler, but I believe it`s a bit 
> easier to understand like this, using some "temporary" commits approach.
> 
> Here`s our starting position:
> 
> (0) ---X1---o---o---o---o---o---X2 (master)
>|\
>| A1---A2---A3
>| \
>|  M (topic)
>| /
>\-B1---B2---B3
> 
> 
> Now, we want to rebase merge commit M from X1 onto X2. First, rebase
> merge commit parents as usual:
> 
> (1) ---X1---o---o---o---o---o---X2
>|\   |\
>| A1---A2---A3   | A1'--A2'--A3'
>| \  |
>|  M |
>| /  |
>\-B1---B2---B3   \-B1'--B2'--B3'
> 
> 
> That was commonly understandable part.

Good. Let's assume that I want to do this interactively (because let's
face it, rebase is boring unless we shake up things a little). And let's
assume that A1 is my only change to the README, and that I realized that
it was incorrect and I do not want the world to see it, so I drop A1'.

Let's see how things go from here:

> Now, for "rebasing" the merge commit (keeping possible amendments), we
> do some extra work. First, we make two temporary commits on top of old
> merge parents, by using exact tree (snapshot) of commit M:
> 
> (2) ---X1---o---o---o---o---o---X2
>|\   |\
>| A1---A2---A3---U1  | A1'--A2'--A3'
>| \  |
>|  M |
>| /  |
>\-B1---B2---B3---U2  \-B1'--B2'--B3'

Okay, everything would still be the same except that I still have dropped
A1'.

> So here, in terms of _snapshots_ (trees, not diffs), U1 = U2 = M.
> 
> Now, we rebase these temporary commits, too:
> 
> (3) ---X1---o---o---o---o---o---X2
>|\   |\
>| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
>| \  |
>|  M |
>| /  |
>\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'

I still want to drop A1 in this rebase, so A1' is still missing.

And now it starts to get interesting.

The diff between A3 and U1 does not touch the README, of course, as I said
that only A1 changed the README.  But the diff between B3 and U2 does
change the README, thanks to M containing A1 change.

Therefore, the diff between B3' and U2' will also have this change to the
README. That change that I wanted to drop.

> As a next step, we merge these temporary commits to produce our
> "rebased" merged commit M:
> 
> (4) ---X1---o---o---o---o---o---X2
>|\   |\
>| A1---A2---A3---U1  | A1'--A2'--A3'--U1'
>| \  |  \
>|  M |   M'
>| /  |  /
>\-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'

And here, thanks to B3'..U2' changing the README, M' will also have that
change that I wanted to see dropped.

Note that A1' is still dropped in my example.

> Finally, we drop temporary commits, and record rebased commits A3' 
> and B3' as our "rebased" merge commit parents instead (merge commit 
> M' keeps its same tree/snapshot state, just gets parents replaced):
> 
> (5) ---X1---o---o---o---o---o---X2
>|\   |\
>| A1---A2---A3---U1  | A1'--A2'--A3'
>| \  | \
>|  M |  M'
>| /  | /
>\-B1---B2---B3---U2  \-B1'--B2'--B3'

Now, thanks to U2' being dropped (and A1' *still* being 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

On 27/02/2018 19:55, Igor Djordjevic wrote:
> 
> It would be more along the lines of "(1) rebase old merge commit parents, 
> (2) generate separate diff between old merge commit and each of its 
> parents, (3) apply each diff to their corresponding newly rebased 
> parent respectively (as a temporary commit, one per rebased parent), 
> (4) merge these temporary commits to generate 'rebased' merge commit, 
> (5) drop temporary commits, recording their parents as parents of 
> 'rebased' merge commit (instead of dropped temporary commits)".
> 
> Implementation wise, steps (2) and (3) could also be done by simply 
> copying old merge commit _snapshot_ on top of each of its parents as 
> a temporary, non-merge commit, then rebasing (cherry-picking) these 
> temporary commits on top of their rebased parent commits to produce 
> rebased temporary commits (to be merged for generating 'rebased' 
> merge commit in step (4)).

For those still tagging along (and still confused), here are some 
diagrams (following what Sergey originally described). Note that 
actual implementation might be even simpler, but I believe it`s a bit 
easier to understand like this, using some "temporary" commits approach.

Here`s our starting position:

(0) ---X1---o---o---o---o---o---X2 (master)
   |\
   | A1---A2---A3
   | \
   |  M (topic)
   | /
   \-B1---B2---B3


Now, we want to rebase merge commit M from X1 onto X2. First, rebase
merge commit parents as usual:

(1) ---X1---o---o---o---o---o---X2
   |\   |\
   | A1---A2---A3   | A1'--A2'--A3'
   | \  |
   |  M |
   | /  |
   \-B1---B2---B3   \-B1'--B2'--B3'


That was commonly understandable part. Now, for "rebasing" the merge 
commit (keeping possible amendments), we do some extra work. First, 
we make two temporary commits on top of old merge parents, by using 
exact tree (snapshot) of commit M:

(2) ---X1---o---o---o---o---o---X2
   |\   |\
   | A1---A2---A3---U1  | A1'--A2'--A3'
   | \  |
   |  M |
   | /  |
   \-B1---B2---B3---U2  \-B1'--B2'--B3'


So here, in terms of _snapshots_ (trees, not diffs), U1 = U2 = M.

Now, we rebase these temporary commits, too:

(3) ---X1---o---o---o---o---o---X2
   |\   |\
   | A1---A2---A3---U1  | A1'--A2'--A3'--U1'
   | \  |
   |  M |
   | /  |
   \-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'


As a next step, we merge these temporary commits to produce our 
"rebased" merged commit M:

(4) ---X1---o---o---o---o---o---X2
   |\   |\
   | A1---A2---A3---U1  | A1'--A2'--A3'--U1'
   | \  |  \
   |  M |   M'
   | /  |  /
   \-B1---B2---B3---U2  \-B1'--B2'--B3'--U2'


Finally, we drop temporary commits, and record rebased commits A3' 
and B3' as our "rebased" merge commit parents instead (merge commit 
M' keeps its same tree/snapshot state, just gets parents replaced):

(5) ---X1---o---o---o---o---o---X2
   |\   |\
   | A1---A2---A3---U1  | A1'--A2'--A3'
   | \  | \
   |  M |  M'
   | /  | /
   \-B1---B2---B3---U2  \-B1'--B2'--B3'


And that`s it, our merge commit M has been "rebased" to M' :)

(6) ---X1---o---o---o---o---o---X2 (master)
|\
| A1'--A2'--A3'
| \
|  M' (topic)
| /
\-B1'--B2'--B3'


Important thing to note here is that in our step (3) above, still in 
terms of trees/snapshots (not diffs), U1' could still be equal to 
U2', produced merge commit M' tree thus being equal to both of them 
as well (merge commit introducing no changes to either of its 
parents, originally described by Sergey as "angel merge").

But it doesn`t have to be so - if any of the rebased commits A1 to A3 
or B1 to B3 was dropped or modified (or extra commits added, even), 
that would influence the trees (snapshots) produced after rebasing U1 
and U2 to U1' and U2', final merge M' reflecting all these changes as 
well, besides keeping original merge commit M amendments (preserving 
"evil merge").


Well, that`s some theory, now to hopefully confirm/test/polish all 
this... or trash it, if flawed beyond correction :P

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Dscho,

On 27/02/2018 17:21, Johannes Schindelin wrote:
> 
> Do you have any way to describe the idea in a simple, 3-5 lines long 
> paragraph?
> 
> So far, I just know that it is some sort of confusing criss-cross 
> cherry-picking and merging and stuff, but nothing in those steps
> shouts out to me what the *idea* is.
> 
> If it would be something like "recreate the old merge, with merge 
> conflicts and all, then generate the diff to the actual tree of the
> merge commit, then apply that to the newly-generated merge", I would
> understand.

It would be more along the lines of "(1) rebase old merge commit parents, 
(2) generate separate diff between old merge commit and each of its 
parents, (3) apply each diff to their corresponding newly rebased 
parent respectively (as a temporary commit, one per rebased parent), 
(4) merge these temporary commits to generate 'rebased' merge commit, 
(5) drop temporary commits, recording their parents as parents of 
'rebased' merge commit (instead of dropped temporary commits)".

Implementation wise, steps (2) and (3) could also be done by simply 
copying old merge commit _snapshot_ on top of each of its parents as 
a temporary, non-merge commit, then rebasing (cherry-picking) these 
temporary commits on top of their rebased parent commits to produce 
rebased temporary commits (to be merged for generating 'rebased' 
merge commit in step (4)).

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Junio C Hamano]

Sergey Organov  writes:

> You've already bit this poor thingy to death. Please rather try your
> teeth on the proposed Trivial Merge (TM) method.

Whatever you do, do *NOT* call any part of your proposal "trivial
merge", unless you are actually using the term to mean what Git
calls "trivial merge".  The phrase has an established meaning in Git
and your attempt to abuse it to mean something entirely different is
adding unnecessary hindrance for other people to understand what you
want to perform.

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Jake,

On Mon, 26 Feb 2018, Jacob Keller wrote:

> On Mon, Feb 26, 2018 at 4:07 PM, Johannes Schindelin
>  wrote:
> >
> > On Tue, 20 Feb 2018, Igor Djordjevic wrote:
> >
> >> I`m really interested in this topic, which seems to (try to) address the
> >> only "bad feeling" I had with rebasing merges - being afraid of silently
> >> losing amendments by actually trying to "replay" the merge (where
> >> additional and possibly important context is missing), instead of really
> >> "rebasing" it (somehow).
> >
> > If those amendments are what you are worried about, why not address them
> > specifically?
> >
> > In other words, rather than performing the equivalent of
> >
> > git show ^! | git apply
> >
> > (which would of course completely ignore if the rewritten ^2
> > dropped a patch, amended a patch, or even added a new patch), what you
> > really want is to figure out what changes the user made when merging, and
> > what merge strategy was used to begin with.
> >
> > To see what I mean, look at the output of `git show 0fd90daba8`: it shows
> > how conflicts were resolved. By necessity, this is more complicated than a
> > simple diff: it is *not* as simple as taking a diff between two revisions
> > and applying that diff to a third revision. There were (at least) three
> > revisions involved in the original merge commit, and recreating that merge
> > commit faithfully means to represent the essence of the merge commit
> > faithfully enough to be able to replay it on a new set of at least three
> > revisions.  That can be simplified to two-way diffs only in very, very
> > special circumstances, and in all other cases this simplification will
> > simply fall on its nose.
> >
> > If the proposed solution was to extend `git apply` to process combined
> > diffs, I would agree that we're on to something. That is not the proposed
> > solution, though.
> >
> > In short: while I am sympathetic to the desire to keep things simple,
> > the idea to somehow side-step replaying the original merge seems to be
> > *prone* to be flawed. Any system that cannot accommodate
> > dropped/changed/added commits on either side of a merge is likely to be
> > too limited to be useful.
> >
> 
> 
> The reason Sergey's solution works is because he cherry picks the
> merge using each parent first, and then merges the result of those. So
> each branch of the merge gets one, and then you merge the result of
> those cherry-picks. This preservers amendments and changes properly,
> and should result in a good solution.

I saw your patch trying to add a minimal example, and I really want to run
away screaming.

Do you have any way to describe the idea in a simple, 3-5 lines long
paragraph?

So far, I just know that it is some sort of confusing criss-cross
cherry-picking and merging and stuff, but nothing in those steps shouts
out to me what the *idea* is.

If it would be something like "recreate the old merge, with merge
conflicts and all, then generate the diff to the actual tree of the merge
commit, then apply that to the newly-generated merge", I would understand.

I would still suspect that -s ours would be a hard nut for that method,
but I would understand that idea.

Thanks,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin  writes:
> Hi Buga,
>
> On Tue, 20 Feb 2018, Igor Djordjevic wrote:
>
>> I`m really interested in this topic, which seems to (try to) address the
>> only "bad feeling" I had with rebasing merges - being afraid of silently
>> losing amendments by actually trying to "replay" the merge (where
>> additional and possibly important context is missing), instead of really
>> "rebasing" it (somehow).
>
> If those amendments are what you are worried about, why not address them
> specifically?
>
> In other words, rather than performing the equivalent of
>
>   git show ^! | git apply
>
> (which would of course completely ignore if the rewritten ^2
> dropped a patch, amended a patch, or even added a new patch),

You've already bit this poor thingy to death. Please rather try your
teeth on the proposed Trivial Merge (TM) method. Sorry, you will need to
actually read the proposal should you decide to.

> what you really want is to figure out what changes the user made when
> merging,

Exactly! It's all and only about changes, I'm glad you are starting to
get it.

> and what merge strategy was used to begin with.

No. It's in Git core philosophy to store only result, independent on the
way the result has been achieved. Only result matters. I could have got
merge result out of thin air, and Git won't care, and will store it for
me safely.

The proposed (TM) method will ensure it's still stored safely after
rebasing.

> To see what I mean, look at the output of `git show 0fd90daba8`: it shows
> how conflicts were resolved. By necessity, this is more complicated than a
> simple diff: it is *not* as simple as taking a diff between two revisions
> and applying that diff to a third revision. There were (at least) three
> revisions involved in the original merge commit, and recreating that merge
> commit faithfully means to represent the essence of the merge commit
> faithfully enough to be able to replay it on a new set of at least three
> revisions.

Even better: we have at least 3 source revisions and 2 new revisions to
base 3-rd new revision upon. Proposed (TM) method will use all of them.

> That can be simplified to two-way diffs only in very, very special
> circumstances, and in all other cases this simplification will simply
> fall on its nose.

No, Done Right (TM) it won't fall as N-way-diff is nothing more than N-1
simple diffs, trivially combined.

> If the proposed solution was to extend `git apply` to process combined
> diffs, I would agree that we're on to something. That is not the proposed
> solution, though.

I'm glad you finally got the point. Proposed (TM) method effectively
'git apply' combined diffs, yes. I just defined it in terms of commits
rather than diffs, as it's simpler to understand and to reason about it
that way.

>
> In short: while I am sympathetic to the desire to keep things simple,

Proposed (TM) method is complex enough to solve the problem at hand. No
need to ask for more complexity.

> the idea to somehow side-step replaying the original merge seems to be
> *prone* to be flawed.

Exactly! That's why current approach to rebase merges is flawed, and
that's why proposed (TM) method does reply entire original merges.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Jacob Keller]

On Mon, Feb 26, 2018 at 4:07 PM, Johannes Schindelin
 wrote:
> Hi Buga,
>
> On Tue, 20 Feb 2018, Igor Djordjevic wrote:
>
>> I`m really interested in this topic, which seems to (try to) address the
>> only "bad feeling" I had with rebasing merges - being afraid of silently
>> losing amendments by actually trying to "replay" the merge (where
>> additional and possibly important context is missing), instead of really
>> "rebasing" it (somehow).
>
> If those amendments are what you are worried about, why not address them
> specifically?
>
> In other words, rather than performing the equivalent of
>
> git show ^! | git apply
>
> (which would of course completely ignore if the rewritten ^2
> dropped a patch, amended a patch, or even added a new patch), what you
> really want is to figure out what changes the user made when merging, and
> what merge strategy was used to begin with.
>
> To see what I mean, look at the output of `git show 0fd90daba8`: it shows
> how conflicts were resolved. By necessity, this is more complicated than a
> simple diff: it is *not* as simple as taking a diff between two revisions
> and applying that diff to a third revision. There were (at least) three
> revisions involved in the original merge commit, and recreating that merge
> commit faithfully means to represent the essence of the merge commit
> faithfully enough to be able to replay it on a new set of at least three
> revisions.  That can be simplified to two-way diffs only in very, very
> special circumstances, and in all other cases this simplification will
> simply fall on its nose.
>
> If the proposed solution was to extend `git apply` to process combined
> diffs, I would agree that we're on to something. That is not the proposed
> solution, though.
>
> In short: while I am sympathetic to the desire to keep things simple,
> the idea to somehow side-step replaying the original merge seems to be
> *prone* to be flawed. Any system that cannot accommodate
> dropped/changed/added commits on either side of a merge is likely to be
> too limited to be useful.
>


The reason Sergey's solution works is because he cherry picks the
merge using each parent first, and then merges the result of those. So
each branch of the merge gets one, and then you merge the result of
those cherry-picks. This preservers amendments and changes properly,
and should result in a good solution.

I agree that making "git apply" work with combined diffs could also be
another solution, but it may be trickier.

If this *doesn't* work, a test case showing that it doesn't work would
be appreciated. I'm hoping to be able to put together something soon,
but I haven't had time due to $dayjob.

> Ciao,
> Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin  writes:
> Hi Buga,
>
> On Tue, 20 Feb 2018, Igor Djordjevic wrote:
>
>> I`m really interested in this topic, which seems to (try to) address the
>> only "bad feeling" I had with rebasing merges - being afraid of silently
>> losing amendments by actually trying to "replay" the merge (where
>> additional and possibly important context is missing), instead of really
>> "rebasing" it (somehow).

[...]

> In short: while I am sympathetic to the desire to keep things simple,
> the idea to somehow side-step replaying the original merge seems to be
> *prone* to be flawed.

The proposed (TM) solution does replay the original merge.

> Any system that cannot accommodate dropped/changed/added commits on
> either side of a merge is likely to be too limited to be useful.

I believe the proposed (TM) solution handles all that nicely. It does
accommodate dropped/changed/added commits on either side of a merge,
symmetrically, and never silently drops user modifications.

If you think (TM) is flawed, please give us a test-case.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Buga,

On Tue, 20 Feb 2018, Igor Djordjevic wrote:

> I`m really interested in this topic, which seems to (try to) address the
> only "bad feeling" I had with rebasing merges - being afraid of silently
> losing amendments by actually trying to "replay" the merge (where
> additional and possibly important context is missing), instead of really
> "rebasing" it (somehow).

If those amendments are what you are worried about, why not address them
specifically?

In other words, rather than performing the equivalent of

git show ^! | git apply

(which would of course completely ignore if the rewritten ^2
dropped a patch, amended a patch, or even added a new patch), what you
really want is to figure out what changes the user made when merging, and
what merge strategy was used to begin with.

To see what I mean, look at the output of `git show 0fd90daba8`: it shows
how conflicts were resolved. By necessity, this is more complicated than a
simple diff: it is *not* as simple as taking a diff between two revisions
and applying that diff to a third revision. There were (at least) three
revisions involved in the original merge commit, and recreating that merge
commit faithfully means to represent the essence of the merge commit
faithfully enough to be able to replay it on a new set of at least three
revisions.  That can be simplified to two-way diffs only in very, very
special circumstances, and in all other cases this simplification will
simply fall on its nose.

If the proposed solution was to extend `git apply` to process combined
diffs, I would agree that we're on to something. That is not the proposed
solution, though.

In short: while I am sympathetic to the desire to keep things simple,
the idea to somehow side-step replaying the original merge seems to be
*prone* to be flawed. Any system that cannot accommodate
dropped/changed/added commits on either side of a merge is likely to be
too limited to be useful.

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Igor,

Igor Djordjevic  writes:

> Hi Sergey,
>

[...]

>
> Even though this behavior is known and documented, it still left some 
> to be desired.
>
> Might be I`m missing something, but so far I like how described 
> approach just "feels right" (to me, for now), being really simple, 
> yet robust :)
>
> As my humble contribution to the cause and discussion itself, I`m 
> providing possibly naive, yet simple demonstration script[1], and 
> clearly showing where current `git rebase --preserve-merges` is 
> lacking (in my opinion, even though being expected and documented), 
> and how your proposal seems to improve the situation here.

Thanks for the test-case, -- it's nice to see this thingy working!

>
> Thanks for your thoughts, and hoping to see this going somewhere :)

So do I. Even if nobody volunteers to adopt it, I hope I'll be able to
implement something myself, not very soon though.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 16/02/2018 14:08, Sergey Organov wrote:
> 
> By accepting the challenges raised in recent discussion of advanced
> support for history rebasing and editing in Git, I hopefully figured out
> a clean and elegant method of rebasing merges that I think is "The Right
> Way (TM)" to perform this so far troublesome operation. ["(TM)" here has
> second meaning: a "Trivial Merge (TM)", see below.]
> 
> Let me begin by outlining the method in git terms, and special thanks
> here must go to "Johannes Sixt"  for his original bright
> idea to use "cherry-pick -m1" to rebase merge commits.
> 
> End of preface -- here we go.
> 
> Given 2 original branches, b1 and b2, and a merge commit M that joins
> them, suppose we've already rebased b1 to b1', and b2 to b2'. Suppose
> also that B1' and B2' happen to be the tip commits on b1' and b2',
> respectively.
> 
> To produce merge commit M' that joins b1' and b2', the following
> operations will suffice:
> 
> 1. Checkout b2' and cherry-pick -m2 M, to produce U2' (and new b2').
> 2. Checkout b1' and cherry-pick -m1 M, to produce U1' (and new b1').
> 3. Merge --no-ff new b2' to current new b1', to produce UM'.
> 4. Get rid of U1' and U2' by re-writing parent references of UM' from
>U1' and U2' to  B1' and B2', respectively, to produce M'.
> 5. Mission complete.
> 
> Let's now see why and how the method actually works.
> 
> Firs off, let me introduce you to my new friend, the Trivial Merge, or
> (TM) for short. By definition, (TM) is a merge that introduces
> absolutely no differences to the sides of the merge. (I also like to
> sometimes call him "Angel Merge", both as the most beautiful of all
> merges, and as direct antithesis to "evil merge".)
> 
> One very nice thing about (TM) is that to safely rebase it, it suffices
> to merge its (rebased) parents. It is safe in this case, as (TM) itself
> doesn't posses any content changes, and thus none could be missed by
> replacing it with another merge commit.
> 
> I bet most of us have never seen (TM) in practice though, so let's see
> how (TM) can help us handle general case of some random merge. What I'm
> going to do is to create a virtual (TM) and see how it goes from there.
> 
> Let's start with this history:
> 
>   M
>  / \
> B1  B2
> 
> And let's transform it to the following one, contextually equivalent to
> the original, by introducing 2 simple utility commits U1 and U2, and a
> new utility merge commit UM:
> 
>   UM
>  /  \
> U1   U2
> ||
> B1   B2
> 
> Here content of any of the created UM, U1, and U2 is the same, and is
> exact copy of original content of M. I.e., provided [A] denotes
> "content of commit A", we have:
> 
> [UM] = [U1] = [U2] = [M]
> 
> Stress again how these changes to the history preserve the exact content
> of the original merge ([UM] = [M]), and how U1 an U2 represent content
> changes due to merge on either side[*], and how neither preceding nor
> subsequent commits content would be affected by the change of
> representation.
> 
> Now observe that as [U1] = [UM], and [U2] = [UM], the UM happens to be
> exactly our new friend -- the "Trivial Merge (TM)" his true self,
> introducing zero changes to content.
> 
> Next we rebase our new representation of the history and we get:
> 
>   UM'
>  /  \
> U1'  U2'
> ||
> B1'  B2'
> 
> Here UM' is bare merge of U1' and U2', in exact accordance with the
> method of rebasing a (TM) we've already discussed above, and U1' and U2'
> are rebased versions of U1 and U2, obtained by usual rebasing methods
> for non-merge commits.
> 
> (Note, however, that at this point UM' is not necessarily a (TM)
> anymore, so in real implementation it may make sense to check if UM' is
> not a (TM) and stop for possible user amendment.)
> 
> Finally, to get to our required merge commit M', we get the content of
> UM' and record two actual parents of the merge:
> 
>   M'
>  / \
> B1' B2'
> 
> Where [M'] = [UM'].
> 
> That's it. Mission complete.
> 
> I expect the method to have the following nice features:
> 
> - it carefully preserves user changes by rebasing the merge commit
> itself, in a way that is semantically similar to rebasing simple
> (non-merge) commits, yet it allows changes made to branches during
> history editing to propagate over corresponding merge commit that joins
> the branches, even automatically when the changes don't conflict, as
> expected.
> 
> - it has provision for detection of even slightest chances of ending up
> with surprising merge (just check if UM' is still (TM)), so that
> implementation could stop for user inspection and amendment when
> appropriate, yet it is capable of handling trivial cases smoothly and
> automatically.
> 
> - it never falls back to simple invocation of merge operation on rebased
> original branches themselves, thus avoiding the problem of lack of
> knowledge of how the merge at hand has been performed in the first
> place. It doesn't prevent implementation from letting user to manually
> 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Jake,

Jacob Keller  writes:
> On Fri, Feb 16, 2018 at 5:08 AM, Sergey Organov  wrote:
>> Hi,
>>
>> By accepting the challenges raised in recent discussion of advanced
>> support for history rebasing and editing in Git, I hopefully figured out
>> a clean and elegant method of rebasing merges that I think is "The Right
>> Way (TM)" to perform this so far troublesome operation. ["(TM)" here has
>> second meaning: a "Trivial Merge (TM)", see below.]
>>
>> Let me begin by outlining the method in git terms, and special thanks
>> here must go to "Johannes Sixt"  for his original bright
>> idea to use "cherry-pick -m1" to rebase merge commits.
>>
>> End of preface -- here we go.
>>
>
> I hope to take a more detailed look at this, also possibly with some
> attempts at re-creating the process by hand to see it in practice.

Thank you for your interest and for the review, and yes, some testing is
what the idea desperately needs. Unfortunately I don't have much time
for it right now, nor am I fluent enough in git internals to actually
make even a raw prototype really soon, sorry. Anybody who is interested
is very welcome to volunteer!

[...]
>
> This might be a bit tricky for a user to understand what the process
> is, especially if they don't understand how it's creating special U1'
> and U2' commits. However, it *is* the cleanest method I've either seen
> or thought of for presenting the conflict to the user.
>
[...]
>> - it appears shiny to the point that it will likely be able to handle
>> even darkest evil merges nicely, no special treatment required.
>>
>
> Yep, and I like that it has a pretty reasonable way of presenting
> conflicts for resolution. It may be a bit tricky to explain the use of
> the intermittent commits U1' and U2' though.

Yeah, I see how all this sends somewhat unique challenges to the
implementation to get user interaction in case of conflicts right, even
though all the basic concepts are old buddies and should be familiar to
the user.

That said, the recursive merge strategy comes to mind, where creating
virtual merge base may itself cause conflicts, so something similar
enough is likely to already exist.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Jacob Keller]

On Fri, Feb 16, 2018 at 5:08 AM, Sergey Organov  wrote:
> Hi,
>
> By accepting the challenges raised in recent discussion of advanced
> support for history rebasing and editing in Git, I hopefully figured out
> a clean and elegant method of rebasing merges that I think is "The Right
> Way (TM)" to perform this so far troublesome operation. ["(TM)" here has
> second meaning: a "Trivial Merge (TM)", see below.]
>
> Let me begin by outlining the method in git terms, and special thanks
> here must go to "Johannes Sixt"  for his original bright
> idea to use "cherry-pick -m1" to rebase merge commits.
>
> End of preface -- here we go.
>

I hope to take a more detailed look at this, also possibly with some
attempts at re-creating the process by hand to see it in practice.

> Given 2 original branches, b1 and b2, and a merge commit M that joins
> them, suppose we've already rebased b1 to b1', and b2 to b2'. Suppose
> also that B1' and B2' happen to be the tip commits on b1' and b2',
> respectively.
>
> To produce merge commit M' that joins b1' and b2', the following
> operations will suffice:
>
> 1. Checkout b2' and cherry-pick -m2 M, to produce U2' (and new b2').
> 2. Checkout b1' and cherry-pick -m1 M, to produce U1' (and new b1').
> 3. Merge --no-ff new b2' to current new b1', to produce UM'.
> 4. Get rid of U1' and U2' by re-writing parent references of UM' from
>U1' and U2' to  B1' and B2', respectively, to produce M'.
> 5. Mission complete.
>

Seems pretty straight forward, go to each branch and cherry-pick the
merge respective to its relative parent, and then finally re-merge
everything, and consume the intermittent commits.

> Let's now see why and how the method actually works.
>
> Firs off, let me introduce you to my new friend, the Trivial Merge, or
> (TM) for short. By definition, (TM) is a merge that introduces
> absolutely no differences to the sides of the merge. (I also like to
> sometimes call him "Angel Merge", both as the most beautiful of all
> merges, and as direct antithesis to "evil merge".)
>
> One very nice thing about (TM) is that to safely rebase it, it suffices
> to merge its (rebased) parents. It is safe in this case, as (TM) itself
> doesn't posses any content changes, and thus none could be missed by
> replacing it with another merge commit.
>
> I bet most of us have never seen (TM) in practice though, so let's see
> how (TM) can help us handle general case of some random merge. What I'm
> going to do is to create a virtual (TM) and see how it goes from there.
>
> Let's start with this history:
>
>   M
>  / \
> B1  B2
>
> And let's transform it to the following one, contextually equivalent to
> the original, by introducing 2 simple utility commits U1 and U2, and a
> new utility merge commit UM:
>
>   UM
>  /  \
> U1   U2
> ||
> B1   B2
>
> Here content of any of the created UM, U1, and U2 is the same, and is
> exact copy of original content of M. I.e., provided [A] denotes
> "content of commit A", we have:
>
> [UM] = [U1] = [U2] = [M]
>
> Stress again how these changes to the history preserve the exact content
> of the original merge ([UM] = [M]), and how U1 an U2 represent content
> changes due to merge on either side[*], and how neither preceding nor
> subsequent commits content would be affected by the change of
> representation.
>
> Now observe that as [U1] = [UM], and [U2] = [UM], the UM happens to be
> exactly our new friend -- the "Trivial Merge (TM)" his true self,
> introducing zero changes to content.
>
> Next we rebase our new representation of the history and we get:
>
>   UM'
>  /  \
> U1'  U2'
> ||
> B1'  B2'
>
> Here UM' is bare merge of U1' and U2', in exact accordance with the
> method of rebasing a (TM) we've already discussed above, and U1' and U2'
> are rebased versions of U1 and U2, obtained by usual rebasing methods
> for non-merge commits.
>
> (Note, however, that at this point UM' is not necessarily a (TM)
> anymore, so in real implementation it may make sense to check if UM' is
> not a (TM) and stop for possible user amendment.)
>

This might be a bit tricky for a user to understand what the process
is, especially if they don't understand how it's creating special U1'
and U2' commits. However, it *is* the cleanest method I've either seen
or thought of for presenting the conflict to the user.

> Finally, to get to our required merge commit M', we get the content of
> UM' and record two actual parents of the merge:
>
>   M'
>  / \
> B1' B2'
>
> Where [M'] = [UM'].
>
> That's it. Mission complete.
>
> I expect the method to have the following nice features:
>
> - it carefully preserves user changes by rebasing the merge commit
> itself, in a way that is semantically similar to rebasing simple
> (non-merge) commits, yet it allows changes made to branches during
> history editing to propagate over corresponding merge commit that joins
> the branches, even automatically when the changes don't conflict, as
> expected.

[RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi,

By accepting the challenges raised in recent discussion of advanced
support for history rebasing and editing in Git, I hopefully figured out
a clean and elegant method of rebasing merges that I think is "The Right
Way (TM)" to perform this so far troublesome operation. ["(TM)" here has
second meaning: a "Trivial Merge (TM)", see below.]

Let me begin by outlining the method in git terms, and special thanks
here must go to "Johannes Sixt"  for his original bright
idea to use "cherry-pick -m1" to rebase merge commits.

End of preface -- here we go.

Given 2 original branches, b1 and b2, and a merge commit M that joins
them, suppose we've already rebased b1 to b1', and b2 to b2'. Suppose
also that B1' and B2' happen to be the tip commits on b1' and b2',
respectively.

To produce merge commit M' that joins b1' and b2', the following
operations will suffice:

1. Checkout b2' and cherry-pick -m2 M, to produce U2' (and new b2').
2. Checkout b1' and cherry-pick -m1 M, to produce U1' (and new b1').
3. Merge --no-ff new b2' to current new b1', to produce UM'.
4. Get rid of U1' and U2' by re-writing parent references of UM' from
   U1' and U2' to  B1' and B2', respectively, to produce M'.
5. Mission complete.

Let's now see why and how the method actually works.

Firs off, let me introduce you to my new friend, the Trivial Merge, or
(TM) for short. By definition, (TM) is a merge that introduces
absolutely no differences to the sides of the merge. (I also like to
sometimes call him "Angel Merge", both as the most beautiful of all
merges, and as direct antithesis to "evil merge".)

One very nice thing about (TM) is that to safely rebase it, it suffices
to merge its (rebased) parents. It is safe in this case, as (TM) itself
doesn't posses any content changes, and thus none could be missed by
replacing it with another merge commit.

I bet most of us have never seen (TM) in practice though, so let's see
how (TM) can help us handle general case of some random merge. What I'm
going to do is to create a virtual (TM) and see how it goes from there.

Let's start with this history:

  M
 / \
B1  B2

And let's transform it to the following one, contextually equivalent to
the original, by introducing 2 simple utility commits U1 and U2, and a
new utility merge commit UM:

  UM
 /  \
U1   U2
||
B1   B2

Here content of any of the created UM, U1, and U2 is the same, and is
exact copy of original content of M. I.e., provided [A] denotes
"content of commit A", we have:

[UM] = [U1] = [U2] = [M]

Stress again how these changes to the history preserve the exact content
of the original merge ([UM] = [M]), and how U1 an U2 represent content
changes due to merge on either side[*], and how neither preceding nor
subsequent commits content would be affected by the change of
representation.

Now observe that as [U1] = [UM], and [U2] = [UM], the UM happens to be
exactly our new friend -- the "Trivial Merge (TM)" his true self,
introducing zero changes to content.

Next we rebase our new representation of the history and we get:

  UM'
 /  \
U1'  U2'
||
B1'  B2'

Here UM' is bare merge of U1' and U2', in exact accordance with the
method of rebasing a (TM) we've already discussed above, and U1' and U2'
are rebased versions of U1 and U2, obtained by usual rebasing methods
for non-merge commits.

(Note, however, that at this point UM' is not necessarily a (TM)
anymore, so in real implementation it may make sense to check if UM' is
not a (TM) and stop for possible user amendment.)

Finally, to get to our required merge commit M', we get the content of
UM' and record two actual parents of the merge:

  M'
 / \
B1' B2'

Where [M'] = [UM'].

That's it. Mission complete.

I expect the method to have the following nice features:

- it carefully preserves user changes by rebasing the merge commit
itself, in a way that is semantically similar to rebasing simple
(non-merge) commits, yet it allows changes made to branches during
history editing to propagate over corresponding merge commit that joins
the branches, even automatically when the changes don't conflict, as
expected.

- it has provision for detection of even slightest chances of ending up
with surprising merge (just check if UM' is still (TM)), so that
implementation could stop for user inspection and amendment when
appropriate, yet it is capable of handling trivial cases smoothly and
automatically.

- it never falls back to simple invocation of merge operation on rebased
original branches themselves, thus avoiding the problem of lack of
knowledge of how the merge at hand has been performed in the first
place. It doesn't prevent implementation from letting user to manually
perform whatever merge she wishes when suspect result is automatically
detected though.

- it extends trivially to octopus merges.

- it appears shiny to the point that it will likely be able to handle
even darkest evil merges nicely, no special treatment required.

Footnote:

[*] We may as well