Troubleshoot clone issue to NFS.

2015-05-21 Thread steve.norman
In setting up some new git servers I was trying to test the performance of some NFS mounted volumes and when compared to local disk (although this is a vitualized server so not truly local) cloning to NFS was taking a long time. Here are some timings: ~ $ time bin/git clone https://github.com/g

Re: Troubleshoot clone issue to NFS.

2015-05-21 Thread Christian Couder
On Thu, May 21, 2015 at 3:13 PM, wrote: [...] > So the questions are: > > 1) Should I expect a clone to NFS to be that much slower? > 2) Is there anything I could do to speed it up (I've tried --bare as that is > what the repositories will be when stored on NFS and there wasn't really a > dif

Re: Troubleshoot clone issue to NFS.

2015-05-21 Thread Duy Nguyen
On Thu, May 21, 2015 at 8:13 PM, wrote: > So there appears to be a change in 1.8.4.2 that made this issue appear for > me. Looking at the release notes the only thing that I can see that might be > related could be: > > * When an object is not found after checking the packfiles and then >

Re: Troubleshoot clone issue to NFS.

2015-05-21 Thread Duy Nguyen
On Thu, May 21, 2015 at 9:31 PM, Duy Nguyen wrote: > In case an object is not found pack directory > is re-read again, which might cause some increased load on nfs. > has_sha1_file() not finding the object should not happen often.. That last statement is probably very wrong, but I have no time to

RE: Troubleshoot clone issue to NFS.

2015-05-21 Thread steve.norman
On Thu, May 21, 2015a at 9:31 PM, Duy Nguyen [mailto:pclo...@gmail.com], did scribble: > > In case an object is not found pack directory is re-read again, which > > might cause some increased load on nfs. > > has_sha1_file() not finding the object should not happen often.. > > That last statement

Re: Troubleshoot clone issue to NFS.

2015-05-21 Thread Duy Nguyen
On Thu, May 21, 2015 at 10:53 PM, wrote: > On Thu, May 21, 2015a at 9:31 PM, Duy Nguyen [mailto:pclo...@gmail.com], did > scribble: >> > In case an object is not found pack directory is re-read again, which >> > might cause some increased load on nfs. >> > has_sha1_file() not finding the object

Re: Troubleshoot clone issue to NFS.

2015-05-22 Thread Jeff King
On Fri, May 22, 2015 at 07:16:54AM +0700, Duy Nguyen wrote: > > Is there anything else I can provide or test? > > In builtin/index-pack.c, replace the line "collision_test_needed = > has_sha1_file(sha1);" with "collision_test_needed = 0;". Security is > compromised but for this test it should be

RE: Troubleshoot clone issue to NFS.

2015-05-22 Thread steve.norman
On Friday, May 22, 2015 @ 8:12 AM Jeff King did scribble: > > In builtin/index-pack.c, replace the line "collision_test_needed = > > has_sha1_file(sha1);" with "collision_test_needed = 0;". Security is > > compromised but for this test it should be ok. Then clone again. I > > hope the new number g

Re: Troubleshoot clone issue to NFS.

2015-05-22 Thread Duy Nguyen
On Fri, May 22, 2015 at 3:35 PM, wrote: > Tested this change on a couple of versions, first of all on the revision > where things go wrong for me: > > ... > > ~ $ time git clone https://github.com/git/git test > > real0m8.263s > user0m10.550s > sys 0m3.763s > > ~ $ time git clone http

Re: Troubleshoot clone issue to NFS.

2015-05-22 Thread Junio C Hamano
Duy Nguyen writes: > Strange. Maybe there is something else... Anyway some numbers from me. > This is nfs3 hosted by Raspberry Pi, accessed over wireless. I just > run index-pack on git.git pack instead of full clone. So the checkout codepath to touch working tree is one difference? Are there ot

RE: Troubleshoot clone issue to NFS.

2015-05-22 Thread steve.norman
On Friday, May 22, 2015 @ 11:06 AM Duy Nguyen did write: > Strange. Maybe there is something else... Anyway some numbers from me. > This is nfs3 hosted by Raspberry Pi, accessed over wireless. I just > run index-pack on git.git pack instead of full clone. > > - v1.8.4.1 34s > - v1.8.4.2 519s (o

Re: Troubleshoot clone issue to NFS.

2015-05-24 Thread Duy Nguyen
On Fri, May 22, 2015 at 2:12 PM, Jeff King wrote: > So I think there are two possibilities for improving this: > > 1. Find places where we expect the object will not exist (like the > collision_test check you pointed out) and use a > "has_sha1_file_fast" that accepts that it may very o

RE: Troubleshoot clone issue to NFS.

2015-06-05 Thread steve.norman
On Sunday, May 24, 2015 @ 10:01 AM Duy Nguyen [mailto:pclo...@gmail.com] did scribble: > In case you want to back away from option 2 because it starts to leak > raciness, which your old commit tried to fix in the first place. I > think the only other place that tests for lots of non-existent loos

Re: Troubleshoot clone issue to NFS.

2015-06-05 Thread Jeff King
On Fri, Jun 05, 2015 at 12:01:16PM +, steve.nor...@thomsonreuters.com wrote: > On Sunday, May 24, 2015 @ 10:01 AM Duy Nguyen [mailto:pclo...@gmail.com] did > scribble: > > > In case you want to back away from option 2 because it starts to leak > > raciness, which your old commit tried to fix

RE: Troubleshoot clone issue to NFS.

2015-06-05 Thread steve.norman
On Friday, June 05, 2015 @ 1:18 PM Jeff King [mailto:p...@peff.net] did scribble: > Sorry, I haven't had a chance to look at it further. It still on my todo > list. My plan is: > > 1. Devise some torture to tests to see whether my patch series is in > fact racy on Linux. > > 2. Assumin

Re: Troubleshoot clone issue to NFS.

2015-06-16 Thread Jeff King
On Fri, Jun 05, 2015 at 08:18:17AM -0400, Jeff King wrote: > On Fri, Jun 05, 2015 at 12:01:16PM +, steve.nor...@thomsonreuters.com > wrote: > > > On Sunday, May 24, 2015 @ 10:01 AM Duy Nguyen [mailto:pclo...@gmail.com] > > did scribble: > > > > > In case you want to back away from option 2