In setting up some new git servers I was trying to test the performance of some NFS mounted volumes and when compared to local disk (although this is a vitualized server so not truly local) cloning to NFS was taking a long time.
Here are some timings: ~ $ time bin/git clone https://github.com/git/git test Cloning into 'test'... remote: Counting objects: 185964, done. remote: Compressing objects: 100% (276/276), done. remote: Total 185964 (delta 203), reused 32 (delta 32), pack-reused 185656 Receiving objects: 100% (185964/185964), 61.42 MiB | 26.16 MiB/s, done. Resolving deltas: 100% (135454/135454), done. Checking connectivity... done. real 0m8.156s user 0m10.569s sys 0m3.857s ~ $ time bin/git clone https://github.com/git/git /sami/test Cloning into '/sami/test'... remote: Counting objects: 185964, done. remote: Compressing objects: 100% (276/276), done. remote: Total 185964 (delta 203), reused 32 (delta 32), pack-reused 185656 Receiving objects: 100% (185964/185964), 61.42 MiB | 10.15 MiB/s, done. Resolving deltas: 100% (135454/135454), done. Checking connectivity... done. Checking out files: 100% (2795/2795), done. real 0m58.685s user 0m12.153s sys 0m7.619s So cloning to NFS is 50 seconds slower on average (I've run this a lot over the last few days and it does appear to be consistent and not a network / github connectivity issue). Tests creating files on the NFS with dd didn't show that much difference to the local disk (and were sometimes quicker). Volume mount differences are: /dev/mapper/rootvg-homelv on /home type ext4 (rw,nodev) nfsserver:/vol/sami/repos on /sami type nfs (rw,bg,nfsvers=3,tcp,hard,nointr,timeo=600,rsize=32768,wsize=32768,addr=10.1.1.1) And there doesn't appear to be any issue with NFS retransmissions: /sami $ nfsstat Client rpc stats: calls retrans authrefrsh 11719847 0 11720190 This morning I did some more digging as when I tried this on a newly build server the NFS times were slower than local disk, but only buy around 6-10 seconds. The new server had git 1.7.1 installed on it compared to 2.4.0 on the machine I've been testing on. So I build a number of versions of git to test each one to try and find the point where it changed: v1.8.0 11.363 s v1.8.3 13.597 s v1.8.4 13.958 s v1.8.4.1 14.563 s v1.8.4.2 1m 0s v1.8.4.3 1m 9s v1.8.4.5 1m 1s v1.8.5 1m 0s v1.8.5.6 1m 0s v1.9.0 1m 38 v2.4.0 58s v2.4.1 58s So there appears to be a change in 1.8.4.2 that made this issue appear for me. Looking at the release notes the only thing that I can see that might be related could be: * When an object is not found after checking the packfiles and then loose object directory, read_sha1_file() re-checks the packfiles to prevent racing with a concurrent repacker; teach the same logic to has_sha1_file(). So the questions are: 1) Should I expect a clone to NFS to be that much slower? 2) Is there anything I could do to speed it up (I've tried --bare as that is what the repositories will be when stored on NFS and there wasn't really a difference). 3) What else can I use in git to compare the performance on local to NFS to see if it is just clones that are affected? 4) I assume I could bisect between 1.8.4.1 and 1.8.4.2 to find exactly where things get worse for me? Thanks for any help, Steve ________________________________ This e-mail is for the sole use of the intended recipient and contains information that may be privileged and/or confidential. If you are not an intended recipient, please notify the sender by return e-mail and delete this e-mail and any attachments. Certain required legal entity disclosures can be accessed on our website.<http://thomsonreuters.com/prof_disclosures/> -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
