Re: [BUG] [git 2.16.1] yeeek ... my files are gone .. by git pull

[Jeff King]

On Mon, Mar 05, 2018 at 03:49:13PM +0100, Johannes Schindelin wrote:

> > I think that is doing the right thing for half of the problem. But
> > there's something else funny where we do not include the "upstream"
> > commits from the split history (i.e., we rebase onto nothing,
> > whereas a normal "git rebase" with a split history will graft the two
> > together).
> 
> Let me ask to make sure I am understanding you correctly. Are you
> referring to "split history" as the case where the commit graph has *two*
> root commits?

Yes, I mean two root commits. But especially when one is in the history
to be rebased, and the other is in the "upstream" history.

So as a concrete example, if I have this repo:

  git init
  >one && git add one && git commit -m one
  git checkout --orphan other
  git mv one two && git commit -m two

and I do this:

  git rebase master

I end up with a two-commit history, with "two" on top of "one". That
makes sense to me. Similarly if I instead do:

  git rebase -i master

the todo list has "pick two", and if I leave it as-is then I get the
same history. But if I do:

  git rebase --preserve-merges master

I end up with a single-commit history, with only commit "one". That's
wrong, because it threw away the history on the "other" branch.

If I apply the patch I showed earlier, then I get a single-branch
history, but this time it contains only "two". That also seems wrong,
because we didn't build on top of "master". I'd expect this command to
give the same results as a non-merge-preserving rebase.

Does that make more sense?

-Peff

Re: [BUG] [git 2.16.1] yeeek ... my files are gone .. by git pull

[Johannes Schindelin]

Hi Peff,

On Wed, 28 Feb 2018, Jeff King wrote:

> On Tue, Feb 27, 2018 at 12:33:56AM +0100, Johannes Schindelin wrote:
> 
> > > So something like this helps:
> > > 
> > > diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> > > index 81c5b42875..71e6cbb388 100644
> > > --- a/git-rebase--interactive.sh
> > > +++ b/git-rebase--interactive.sh
> > > @@ -921,15 +921,20 @@ else
> > >  
> > >   if test -z "$rebase_root"
> > >   then
> > >   preserve=t
> > > + p=
> > >   for p in $(git rev-list --parents -1 $sha1 | cut -d' ' 
> > > -s -f2-)
> > >   do
> > >   if test -f "$rewritten"/$p
> > >   then
> > >   preserve=f
> > >   fi
> > >   done
> > > + if test -z "$p"
> > > + then
> > > + preserve=f
> > > + fi
> > >   else
> > >   preserve=f
> > >   fi
> > >   if test f = "$preserve"
> > > 
> > > Because it at least adds "two" to the list of commits to pick. But
> > > oddly, it picks it directly as a root commit again. Whereas a rebase
> > > without --preserve-merges (and even "-i") picks it on top of commit
> > > "one" (which is what I'd expect).
> > > 
> > > +cc Dscho, as the --preserve-merges guru.
> > 
> > Your analysis makes sense to me. Please note, though, that I would not
> > consider myself a guru on preserve-merges. I think this mode is broken by
> > design (you can blame me if you want).
> 
> I think that is doing the right thing for half of the problem. But
> there's something else funny where we do not include the "upstream"
> commits from the split history (i.e., we rebase onto nothing,
> whereas a normal "git rebase" with a split history will graft the two
> together).

Let me ask to make sure I am understanding you correctly. Are you
referring to "split history" as the case where the commit graph has *two*
root commits?

If so: when you perform a merge-preserving rebase, then those two root
commits will be recreated as new root commits, by design.

The non-merge-preserving mode cannot create two root commits, as it does
not allow for introducing merge commits (and you'd need that to combine
the two strands).

It is quite possible that I misunderstand completely, though. Care to
enlighten me?

Ciao,
Dscho

Re: [BUG] [git 2.16.1] yeeek ... my files are gone .. by git pull

[Jeff King]

On Tue, Feb 27, 2018 at 12:33:56AM +0100, Johannes Schindelin wrote:

> > So something like this helps:
> > 
> > diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> > index 81c5b42875..71e6cbb388 100644
> > --- a/git-rebase--interactive.sh
> > +++ b/git-rebase--interactive.sh
> > @@ -921,15 +921,20 @@ else
> >  
> > if test -z "$rebase_root"
> > then
> > preserve=t
> > +   p=
> > for p in $(git rev-list --parents -1 $sha1 | cut -d' ' 
> > -s -f2-)
> > do
> > if test -f "$rewritten"/$p
> > then
> > preserve=f
> > fi
> > done
> > +   if test -z "$p"
> > +   then
> > +   preserve=f
> > +   fi
> > else
> > preserve=f
> > fi
> > if test f = "$preserve"
> > 
> > Because it at least adds "two" to the list of commits to pick. But
> > oddly, it picks it directly as a root commit again. Whereas a rebase
> > without --preserve-merges (and even "-i") picks it on top of commit
> > "one" (which is what I'd expect).
> > 
> > +cc Dscho, as the --preserve-merges guru.
> 
> Your analysis makes sense to me. Please note, though, that I would not
> consider myself a guru on preserve-merges. I think this mode is broken by
> design (you can blame me if you want).

I think that is doing the right thing for half of the problem. But
there's something else funny where we do not include the "upstream"
commits from the split history (i.e., we rebase onto nothing,
whereas a normal "git rebase" with a split history will graft the two
together).

I haven't figured that part out. But I do kind of wonder if we should
simply punt early and say "what you're doing is crazy and unsupported".
After all, the only known case of this was somebody doing it
accidentally.

-Peff

Re: [BUG] [git 2.16.1] yeeek ... my files are gone .. by git pull

[Johannes Schindelin]

Hi Peff,

On Fri, 23 Feb 2018, Jeff King wrote:

> On Fri, Feb 23, 2018 at 06:29:55AM +0100, "Marcel 'childNo͡.de' Trautwein" 
> wrote:
> 
> > shows me a quite different behavior, so solely rebase not seems the
> > full problem BUT `--rebase=preserve` will .. o’man , really, is this
> > intended?
> 
> Yeah, the bug seems to be in --preserve-merges. Here's an easier
> reproduction:
> 
>   git init
>   >one && git add one && git commit -m one
>   git checkout --orphan other
>   git mv one two && git commit -m two
> 
>   git rebase --preserve-merges master
> 
> at which point we've dropped commit "two" and its files entirely.
> 
> I don't know much about how preserve-merges works. It looks like in the
> loop around git-rebase--interactive.sh:931 that we mark commit "two"
> with preserve=t, and so we _don't_ add it to the list of commits to
> pick.
> 
> I think that stems from the fact that it has no parent marked to be
> rewritten.
> 
> So something like this helps:
> 
> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> index 81c5b42875..71e6cbb388 100644
> --- a/git-rebase--interactive.sh
> +++ b/git-rebase--interactive.sh
> @@ -921,15 +921,20 @@ else
>  
>   if test -z "$rebase_root"
>   then
>   preserve=t
> + p=
>   for p in $(git rev-list --parents -1 $sha1 | cut -d' ' 
> -s -f2-)
>   do
>   if test -f "$rewritten"/$p
>   then
>   preserve=f
>   fi
>   done
> + if test -z "$p"
> + then
> + preserve=f
> + fi
>   else
>   preserve=f
>   fi
>   if test f = "$preserve"
> 
> Because it at least adds "two" to the list of commits to pick. But
> oddly, it picks it directly as a root commit again. Whereas a rebase
> without --preserve-merges (and even "-i") picks it on top of commit
> "one" (which is what I'd expect).
> 
> +cc Dscho, as the --preserve-merges guru.

Your analysis makes sense to me. Please note, though, that I would not
consider myself a guru on preserve-merges. I think this mode is broken by
design (you can blame me if you want).

Ciao,
Dscho

Re: [BUG] [git 2.16.1] yeeek ... my files are gone .. by git pull

[Jeff King]

On Fri, Feb 23, 2018 at 06:29:55AM +0100, "Marcel 'childNo͡.de' Trautwein" 
wrote:

> shows me a quite different behavior, so solely rebase not seems the full 
> problem
> BUT
> `--rebase=preserve` will .. o’man , really, is this intended?

Yeah, the bug seems to be in --preserve-merges. Here's an easier
reproduction:

  git init
  >one && git add one && git commit -m one
  git checkout --orphan other
  git mv one two && git commit -m two

  git rebase --preserve-merges master

at which point we've dropped commit "two" and its files entirely.

I don't know much about how preserve-merges works. It looks like in the
loop around git-rebase--interactive.sh:931 that we mark commit "two"
with preserve=t, and so we _don't_ add it to the list of commits to
pick.

I think that stems from the fact that it has no parent marked to be
rewritten.

So something like this helps:

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 81c5b42875..71e6cbb388 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -921,15 +921,20 @@ else
 
if test -z "$rebase_root"
then
preserve=t
+   p=
for p in $(git rev-list --parents -1 $sha1 | cut -d' ' 
-s -f2-)
do
if test -f "$rewritten"/$p
then
preserve=f
fi
done
+   if test -z "$p"
+   then
+   preserve=f
+   fi
else
preserve=f
fi
if test f = "$preserve"

Because it at least adds "two" to the list of commits to pick. But
oddly, it picks it directly as a root commit again. Whereas a rebase
without --preserve-merges (and even "-i") picks it on top of commit
"one" (which is what I'd expect).

+cc Dscho, as the --preserve-merges guru.

-Peff

Re: [BUG] [git 2.16.1] yeeek ... my files are gone .. by git pull

[Marcel 'childNo͡.de' Trautwein]

> Am 23.02.2018 um 00:20 schrieb Jonathan Nieder :
> 
> Hi Marcel,
> 
> …
> Sorry, this is not the most helpful reply but:
> 
> Can you describe a reproduction recipe so that I can experience the
> same thing?
> 
> That is:
> 
> 1. steps to reproduce
> 2. expected result
> 3. actual result
> 4. the difference and why it was unexpected
> 

1. steps to reproduce
=
```
Last login: Fri Feb 23 00:33:11 on ttys001
~ PATH variable not enhanced, no applications found in ~/Applications/*-latest


-bash:/Users/marcel:$ mkdir /tmp/$$
change to new directory '/tmp/2608'? [Y/n] 


-bash:/tmp/2608:$ mkdir a.git
change to new directory 'a.git'? [Y/n] 


-bash:/tmp/2608/a.git:$ git init
Initialized empty Git repository in /private/tmp/2608/a.git/.git/


-bash:/tmp/2608/a.git:$ touch foo


-bash:/tmp/2608/a.git:$ git add foo


-bash:/tmp/2608/a.git:$ git commit -m "foo" foo
[master (root-commit) ed191c4] foo
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 foo


-bash:/tmp/2608/a.git:$ cd -
/tmp/2608


-bash:/tmp/2608:$ mkdir b.git
change to new directory 'b.git'? [Y/n] 


-bash:/tmp/2608/b.git:$ git init
Initialized empty Git repository in /private/tmp/2608/b.git/.git/


-bash:/tmp/2608/b.git:$ touch bar


-bash:/tmp/2608/b.git:$ git add bar


-bash:/tmp/2608/b.git:$ git commit -m "bar" bar
[master (root-commit) 80b0355] bar
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 bar


-bash:/tmp/2608/b.git:$ cd -
/tmp/2608


-bash:/tmp/2608:$ git clone a.git c
Cloning into 'c'...
done.


-bash:/tmp/2608:$ cd c


-bash:/tmp/2608/c:$ ll
total 0
drwxr-xr-x  12 marcel  wheel   384B 23 Feb 05:47 .git
-rw-r--r--   1 marcel  wheel 0B 23 Feb 05:47 foo


-bash:/tmp/2608/c:$ git pull ../b.git/
warning: no common commits
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From ../b
 * branchHEAD   -> FETCH_HEAD
Successfully rebased and updated refs/heads/master.


-bash:/tmp/2608/c:$ ll
total 0
drwxr-xr-x  14 marcel  wheel   448B 23 Feb 05:47 .git
-rw-r--r--   1 marcel  wheel 0B 23 Feb 05:47 bar


-bash:/tmp/2608/c:$ git reflog
80b0355 (HEAD -> master) HEAD@{0}: pull ../b.git/: checkout 
80b03552466bc526b1130ce5ca4a991ba31a0546: returning to refs/heads/master
80b0355 (HEAD -> master) HEAD@{1}: pull ../b.git/: checkout 
80b03552466bc526b1130ce5ca4a991ba31a0546
ed191c4 (origin/master, origin/HEAD) HEAD@{2}: clone: from /tmp/2608/a.git


-bash:/tmp/2608/c:$ git remote -v
origin  /tmp/2608/a.git (fetch)
origin  /tmp/2608/a.git (push)


-bash:/tmp/2608/c:$  git log --all --graph --decorate --oneline 
--simplify-by-decoration
* 80b0355 (HEAD -> master) bar
* ed191c4 (origin/master, origin/HEAD) foo
```

2. expected result
==
just an error in case the too trees have no common ancestors

```
-bash:/tmp/2608/c:$ git pull ../b.git/
warning: no common commits
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From ../b
 * branchHEAD   -> FETCH_HEAD
fatal: refusing to merge unrelated histories
```

3. actual result

pulls out, removes all files from the first tree

4. the difference and why it was unexpected
===
I can’t find words on it … it should not work but it did? somehow … with 
unexpected results to my local repository

it somehow seems to be an issue of my config, because resetting it, will not 
allow the pull as expected

```
-bash:/tmp/2608/c:$ GIT_CONFIG_NOSYSTEM=1 HOME=. git config -l
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
core.ignorecase=true
core.precomposeunicode=true
remote.origin.url=/tmp/2608/a.git
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master


-bash:/tmp/2608/c:$ GIT_CONFIG_NOSYSTEM=1 HOME=. git pull ../b.git/
warning: no common commits
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From ../b
 * branchHEAD   -> FETCH_HEAD
fatal: refusing to merge unrelated histories


-bash:/tmp/2608/c:$ git pull ../b.git/
From ../b
 * branchHEAD   -> FETCH_HEAD
Successfully rebased and updated refs/heads/master.
```

the logs tells me he rebases ...
```
-bash:/tmp/2608/c:$ git config -l | grep merge
diff.tool=p4merge
merge.tool=p4merge
merge.branchdesc=true
merge.log=true
branch.autosetupmerge=true
branch.master.merge=refs/heads/master


-bash:/tmp/2608/c:$ git config -l | grep pull
pull.rebase=preserve


-bash:/tmp/2608/c:$ git config -l | grep fetch
fetch.recursesubmodules=true
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
```


> I suspect that this information is in your message, somewhere, but it
> is (understandably) unfocussed and I am having trouble pulling it out.
> 

I’m sorry, 

Re: [BUG] [git 2.16.1] yeeek ... my files are gone .. by git pull

[Jonathan Nieder]

Hi Marcel,

Marcel 'childNo͡.de' Trautwein" wrote:

> I think we have a problem … or at least I had
> and I’m not quite sure if this is „working as designed“
> but I’m sure it „should not work as it did“.
[...]
> I wanted to clone another repository … but yeah … it’s late for me today and 
> I put
> in s.th. `git pull 
> g...@private.gitlab.instance.example.com:aGroup/repository.git`
>
> next … all committed files are zapped and the repository given has
> been checked out in my home directory 勞
>
> what? Shouldn’t this just fail? Why can I pass another remote to pull?

Sorry, this is not the most helpful reply but:

Can you describe a reproduction recipe so that I can experience the
same thing?

That is:

 1. steps to reproduce
 2. expected result
 3. actual result
 4. the difference and why it was unexpected

I suspect that this information is in your message, somewhere, but it
is (understandably) unfocussed and I am having trouble pulling it out.

[...]
> trying to fix this up by doing another pull failed:
> ```
> -bash:$ git remote -v
> origing...@bitbucket.org:childnode/marcel.git (fetch)
> origing...@bitbucket.org:childnode/marcel.git (push)
>
> -bash:$ git pull
> fatal: refusing to merge unrelated histories

Ok, this part is something I might be able to help shed some light on.

Searching for 'unrelated' in "git help pull" finds:

   --allow-unrelated-histories
   By default, git merge command refuses to merge histories that do not
   share a common ancestor. This option can be used to override this
   safety when merging histories of two projects that started their
   lives independently. As that is a very rare occasion, no
   configuration variable to enable this by default exists and will not
   be added.

So that explains the "what" of that error message.

The "why" is a separate question.  Could you share output from

  git log --all --graph --decorate --oneline --simplify-by-decoration

and

  git status

to help us understand your current state?

Also, suggestions for improvements to the 'refusing to merge' message
would be very welcome.

Thanks and hope that helps,
Jonathan

[BUG] [git 2.16.1] yeeek ... my files are gone .. by git pull

[Marcel 'childNo͡.de' Trautwein]

Hi,

I think we have a problem … or at least I had
and I’m not quite sure if this is „working as designed“
but I’m sure it „should not work as it did“.

Because? It pruned a lot of files and even the local repository.
by pull
by giving another repository URL instead of a known remote

While working in a subpath of my homedir
(that is a git repository itself, without any changes in worktree or index: 
https://bitbucket.org/childnode/marcel/ )
I wanted to clone another repository … but yeah … it’s late for me today and I 
put
in s.th. `git pull 
g...@private.gitlab.instance.example.com:aGroup/repository.git`

next … all committed files are zapped and the repository given has been checked 
out in my home directory 勞

what? Shouldn’t this just fail? Why can I pass another remote to pull?

 god any untracked / ignored files are still alive

a, yeh, I’m on a mac 
(for any git configuration … have a look in my repository 
https://bitbucket.org/childnode/marcel/src/88ff8d0c28bb90dfde3aea9e6c39bb551bea8ca8/.gitconfig?at=master=file-view-default

the console out was:
```
-bash:$ git pull g...@private.gitlab.instance.example.com:aGroup/repository.git
Warning: Permanently added the ECDSA host key for IP address '10.1.2.3' to the 
list of known hosts.
warning: no common commits
remote: Counting objects: 2301, done.
remote: Compressing objects: 100% (710/710), done.
remote: Total 2301 (delta 1040), reused 2239 (delta 1004)
Receiving objects: 100% (2301/2301), 405.41 KiB | 635.00 KiB/s, done.
Resolving deltas: 100% (1040/1040), done.
From private.gitlab.instance.example.com:aGroup/repository
 * branchHEAD   -> FETCH_HEAD
Fetching submodule .shapps/willgit
Fetching submodule .vim
Fetching submodule .vim/autoload/pathogen
warning: redirecting to https://github.com/tpope/vim-pathogen.git/
Fetching submodule .vim/bundle/ack
warning: redirecting to https://github.com/mileszs/ack.vim.git/
Fetching submodule .vim/bundle/colors-solarized
warning: redirecting to https://github.com/altercation/vim-colors-solarized.git/
Fetching submodule .vim/bundle/flake8
Fetching submodule .vim/bundle/fugitive
warning: redirecting to https://github.com/tpope/vim-fugitive.git/
Fetching submodule .vim/bundle/kwbdi
warning: redirecting to https://github.com/vim-scripts/kwbdi.vim.git/
Fetching submodule .vim/bundle/markdown
warning: redirecting to https://github.com/tpope/vim-markdown.git/
Fetching submodule .vim/bundle/nerdtree
warning: redirecting to https://github.com/scrooloose/nerdtree.git/
Fetching submodule .vim/bundle/unimpaired
warning: redirecting to https://github.com/tpope/vim-unimpaired.git/
Fetching submodule 
gists/bitbucket/childnode/2015-06-16_G4pLy_prevent-empty-version-comment-in.git
Fetching submodule 
gists/bitbucket/childnode/2015-06-21_kyAAM_plasticscm-addcurrentworkdir-batch-task
Fetching submodule gists/github/childnode/18de20f4448692257aa3e99c8319b70d
Fetching submodule gists/github/childnode/295dbd6e_hasSize.regex
Fetching submodule gists/github/childnode/4a0de936_gradle_buildSrc_dogfood
Fetching submodule gists/github/childnode/66d4b982_git.rebaseAllBranches
Fetching submodule 
gists/github/childnode/6df6d14c_ideaGradleProjectSetupForAdditionalSourceSets
Fetching submodule gists/github/childnode/81ae6468_build_jar_manifest.gradle
Fetching submodule gists/github/childnode/85958ff8_extendedHelp.gradle_secret
Fetching submodule 
gists/github/childnode/88304258_git_deleteAllRemoteBranches.sh
Fetching submodule gists/github/childnode/8f100f90_dockerSaveAllImages.sh
Fetching submodule 
gists/github/childnode/9741c4d1_idea.warnGenerateWorkspace.gradle_secret
Fetching submodule gists/github/childnode/a175d954_ext.props.gradle_secret
Fetching submodule gists/github/childnode/d15cd5e9_atlassian-confluence-config
Fetching submodule gists/github/childnode/d35cf810dd28775ac5c0e491107215fd
Fetching submodule gists/github/childnode/da08e8a6f989ce0f94077ae1a6b1573b
Fetching submodule gists/github/childnode/e7ef876c_html2ical_secret
Fetching submodule gists/github/childnode/eb3199790f2f82785f62c3150f352ede
Successfully rebased and updated refs/heads/master.

```

trying to fix this up by doing another pull failed:
```
-bash:$ git remote -v
origin  g...@bitbucket.org:childnode/marcel.git (fetch)
origin  g...@bitbucket.org:childnode/marcel.git (push)

-bash:$ git pull
fatal: refusing to merge unrelated histories

-bash:$ git pull g...@bitbucket.org:childnode/marcel.git
From bitbucket.org:childnode/marcel
 * branchHEAD   -> FETCH_HEAD
fatal: refusing to merge unrelated histories

```

these messages and the fact that it doesn’t work backward let me think I ran 
into
a collision? really?

revlog looks a bit strange too
```
04f3066 (HEAD -> master) HEAD@{0}: pull 
g...@private.gitlab.instance.example.com:aGroup/repository.git: checkout 
04f3066d03e09323c7341c472be4c45ea5e3a4ff: returning to refs/heads/master
04f3066 (HEAD -> master) HEAD@{1}: pull