Re: Bare repository fetch/push race condition

2017-11-30 Thread Dmitry Neverov
Sorry for misleading subject. It should be "Race condition between pushing to and pushing from a bare repository"

Bare repository fetch/push race condition

2017-11-30 Thread Dmitry Neverov
It looks like there is a race condition between fetch and push in a bare repository in the following setup. There is a bare git repository on a local file system. Some process pushes to this repository via jgit. There is a cron task which pushes this repository to the backup remote repo over ssh.

Re: Git push race condition?

2014-04-11 Thread Matthieu Moy
This finally happened again. Here's what the reflog looks like: 2805f68 master@{0}: push 96eebc0 master@{1}: push 75bd4a6 master@{2}: push abc30da master@{3}: push eba874f master@{4}: push 10981e7 master@{5}: push 76b3957 master@{6}: push 2e3ea06 master@{7}: push 9d4e778 master@{8}:

Re: Git push race condition?

2014-04-10 Thread Scott Sandler
On Tue, Mar 25, 2014 at 10:57 AM, Jeff King p...@peff.net wrote: On Tue, Mar 25, 2014 at 09:45:20AM -0400, Scott Sandler wrote: Version of git on the server? git version 1.8.3-rc0 There was significant work done between v1.8.3 and v1.8.4 on handling races in the ref code. As I said before, I

Re: Git push race condition?

2014-03-25 Thread Scott Sandler
Version of git on the server? git version 1.8.3-rc0 Is there a hook or cron job that updates or gcs this repository or any refs? No. No cron jobs touching the repo at all, and all the hooks are read-only. There are pre-receive hooks that either reject a push or don't based on some checks, there

Re: Git push race condition?

2014-03-25 Thread Matthieu Moy
Scott Sandler scott.m.sand...@gmail.com writes: Is there a hook or cron job that updates or gcs this repository or any refs? No. No cron jobs touching the repo at all, and all the hooks are read-only. If you activated the reflog, you can double-check that. Running git reflog on the server

Re: Git push race condition?

2014-03-25 Thread Scott Sandler
I'm definitely open to the possibility there's a problem with my setup. I've got the reflog on now and will check what that looks like next time the issue happens. So far it looks like you described: b2b202d master@{0}: push 7c01312 master@{1}: push 3635312 master@{2}: push aea29bf master@{3}:

Re: Git push race condition?

2014-03-25 Thread Matthieu Moy
(please, don't top-post on this list) Scott Sandler scott.m.sand...@gmail.com writes: I'd like to see it happen again under these conditions and get that information, then enable receive.denyNonFastForwards explicitly just to be sure it's not force pushes, and see if it still happens. To be

Re: Git push race condition?

2014-03-25 Thread Jeff King
On Tue, Mar 25, 2014 at 09:45:20AM -0400, Scott Sandler wrote: Version of git on the server? git version 1.8.3-rc0 There was significant work done between v1.8.3 and v1.8.4 on handling races in the ref code. As I said before, I don't think the symptoms you are describing are anything we have

Git push race condition?

2014-03-24 Thread Scott Sandler
-forward before running hooks, and that the hooks taking a few seconds creates more opportunity for a race condition to occur. However, after reading http://git.661346.n2.nabble.com/push-race-td7569254.html and doing some of my own testing (creating a hook that runs for 60 seconds and pushing from two

Re: Git push race condition?

2014-03-24 Thread Matthieu Moy
Scott Sandler scott.m.sand...@gmail.com writes: Both pushes are determined to be fast-forwards and both succeed, but B' overwrites B and B is no longer on origin/master. The server does have B in its .git directory but the commit isn't on any branch. Is the reflog enabled on the server? If

Re: Git push race condition?

2014-03-24 Thread Scott Sandler
It's a bare repo and I didn't realize server-side reflogs were a thing. Just ran git config core.logallrefupdates true in the repo on the server which seems to be what I should do to enable that. The server does know about B, it shows up when you do git show B. However git branch --contains B

Re: Git push race condition?

2014-03-24 Thread Ævar Arnfjörð Bjarmason
On Mon, Mar 24, 2014 at 8:18 PM, Scott Sandler scott.m.sand...@gmail.com wrote: I run a private Git repository (using Gitlab) with about 200 users doing about 100 pushes per day. Ditto but about 2x those numbers. error: Ref refs/heads/master is at 4584c1f34e07cea2df6abc8e0d407fe016017130 but

Re: Git push race condition?

2014-03-24 Thread Scott Sandler
Right. Receiving that error is what happens during my testing with a hook that sleeps for 60s, and that outcome makes sense. But whatever is occurring in production must be different, since both users see successful pushes with the first one just being overwritten. On Mon, Mar 24, 2014 at 5:16

Re: Git push race condition?

2014-03-24 Thread Matthieu Moy
Scott Sandler scott.m.sand...@gmail.com writes: It's a bare repo and I didn't realize server-side reflogs were a thing. Just ran git config core.logallrefupdates true in the repo on the server which seems to be what I should do to enable that. That should be it, yes. The server does know

Re: Git push race condition?

2014-03-24 Thread Junio C Hamano
Matthieu Moy matthieu@grenoble-inp.fr writes: What you describe really looks like a force-push, or a hook doing a ref update (e.g. a hook on a dev branch that updates master if the code passes tests or so). ... or a filesystem that is broken. But I thought this is just a plain-vanilla

Re: Git push race condition?

2014-03-24 Thread Jeff King
On Mon, Mar 24, 2014 at 10:16:52PM +0100, Ævar Arnfjörð Bjarmason wrote: error: Ref refs/heads/master is at 4584c1f34e07cea2df6abc8e0d407fe016017130 but expected 61b79b6d35b066d054fb3deab550f1c51598cf5f remote: error: failed to lock refs/heads/master I also see this error once in a

Re: Git push race condition?

2014-03-24 Thread Jeff King
On Mon, Mar 24, 2014 at 03:18:14PM -0400, Scott Sandler wrote: I've noticed that a few times in the past several weeks, we've had events where pushes have been lost when two people pushed at just about the same time. The scenario is that two users both have commits based on commit A, call

Re: Git push race condition?

2014-03-24 Thread Nasser Grainawi
On Mar 24, 2014, at 4:54 PM, Jeff King p...@peff.net wrote: On Mon, Mar 24, 2014 at 03:18:14PM -0400, Scott Sandler wrote: I've noticed that a few times in the past several weeks, we've had events where pushes have been lost when two people pushed at just about the same time. The scenario

Re: push race

2012-10-16 Thread Angelo Borsotti
Hi Jeff, it would be worth to put your description as comments in the code for future reference. Thanks -Angelo -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: push race

2012-10-16 Thread Jeff King
On Tue, Oct 16, 2012 at 05:45:12PM +0700, Nguyen Thai Ngoc Duy wrote: On Tue, Oct 16, 2012 at 12:37 PM, Jeff King p...@peff.net wrote: I suspect a site that is heavy on alternates is invoking the index-pack code path more frequently than necessary (e.g., history gets pushed to one forked

Re: push race

2012-10-16 Thread Junio C Hamano
Nguyen Thai Ngoc Duy pclo...@gmail.com writes: On Tue, Oct 16, 2012 at 12:37 PM, Jeff King p...@peff.net wrote: I suspect a site that is heavy on alternates is invoking the index-pack code path more frequently than necessary (e.g., history gets pushed to one forked repo, then when it goes to

Re: push race

2012-10-16 Thread Jeff King
On Tue, Oct 16, 2012 at 10:21:02AM -0700, Junio C Hamano wrote: I suppose we could do the way unpack-objects does: prefer present objects and drop the new identical ones, no memcmp. Objects that are not bases, or are ref-delta bases, can be safely dropped. ofs-delta bases may lead to

Re: push race

2012-10-15 Thread Matthieu Moy
Angelo Borsotti angelo.borso...@gmail.com writes: the push command checks first if the tips of the branches match those of the remote references, and if it does uploads the snapshot. The update does two things: upload objects to the database, and then update the reference. Adding objects to

Re: push race

2012-10-15 Thread Nguyen Thai Ngoc Duy
On Mon, Oct 15, 2012 at 6:05 PM, Matthieu Moy matthieu@grenoble-inp.fr wrote: Angelo Borsotti angelo.borso...@gmail.com writes: the push command checks first if the tips of the branches match those of the remote references, and if it does uploads the snapshot. The update does two things:

Re: push race

2012-10-15 Thread Ævar Arnfjörð Bjarmason
On Mon, Oct 15, 2012 at 11:14 AM, Angelo Borsotti angelo.borso...@gmail.com wrote: Hello, FWIW we have a lot of lemmings pushing to the same ref all the time at $work, and while I've seen cases where: 1. Two clients try to push 2. They both get the initial lock 3. One of them fails to get

Re: push race

2012-10-15 Thread demerphq
On 15 October 2012 16:09, Ævar Arnfjörð Bjarmason ava...@gmail.com wrote: On Mon, Oct 15, 2012 at 11:14 AM, Angelo Borsotti angelo.borso...@gmail.com wrote: Hello, FWIW we have a lot of lemmings pushing to the same ref all the time at $work, and while I've seen cases where: 1. Two clients

Re: push race

2012-10-15 Thread Marc Branchaud
On 12-10-15 10:09 AM, Ævar Arnfjörð Bjarmason wrote: On Mon, Oct 15, 2012 at 11:14 AM, Angelo Borsotti angelo.borso...@gmail.com wrote: Hello, FWIW we have a lot of lemmings pushing to the same ref all the time at $work, and while I've seen cases where: 1. Two clients try to push 2.

Re: push race

2012-10-15 Thread Angelo Borsotti
Hi Marc, correct, there will be no file overwriting because no files are written on the work tree. I tried to follow the actions of the program, but did not quite catch the 6. you mention. -Angelo -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to

Re: push race

2012-10-15 Thread Jeff King
On Mon, Oct 15, 2012 at 10:29:08AM -0400, Marc Branchaud wrote: Here's a previous discussion of a race in concurrent updates to the same ref, even when the updates are all identical: http://news.gmane.org/find-root.php?group=gmane.comp.version-control.gitarticle=164636 In that thread,

Re: push race

2012-10-15 Thread Jeff King
On Mon, Oct 15, 2012 at 05:50:47PM +0200, Angelo Borsotti wrote: correct, there will be no file overwriting because no files are written on the work tree. I tried to follow the actions of the program, but did not quite catch the 6. you mention. It is the oldval parameter to

Re: push race

2012-10-15 Thread Shawn Pearce
On Mon, Oct 15, 2012 at 11:56 AM, Jeff King p...@peff.net wrote: Right. The only thing that needs locking is the refs, because the object database is add-only for normal operations, and by definition collisions mean you have the same content (or are astronomically unlucky, but your consolation

Re: push race

2012-10-15 Thread Jeff King
On Mon, Oct 15, 2012 at 07:09:52PM -0700, Shawn O. Pearce wrote: On Mon, Oct 15, 2012 at 11:56 AM, Jeff King p...@peff.net wrote: Right. The only thing that needs locking is the refs, because the object database is add-only for normal operations, and by definition collisions mean you have

Re: push race

2012-10-15 Thread Jeff King
On Tue, Oct 16, 2012 at 12:15:21PM +0700, Nguyen Thai Ngoc Duy wrote: On Tue, Oct 16, 2012 at 11:51 AM, Jeff King p...@peff.net wrote: Its worth nothing that a SHA-1 collision can be identified at the server because the server performs a byte-for-byte compare of both copies of the object