Hi guys, Any further interest on this scalability problem or should I move on?
Thanks, Uri On Thu, Nov 8, 2012 at 5:35 PM, Uri Moszkowicz <u...@4refs.com> wrote: > I tried on the local disk as well and it didn't help. I managed to > find a SUSE11 machine and tried it there but no luck so I think we can > eliminate NFS and OS as significant factors now. > > I ran with perf and here's the report: > > ESC[31m 69.07%ESC[m git /lib64/libc-2.11.1.so > [.] memcpy > ESC[31m 12.33%ESC[m git > <prefix>/git-1.8.0.rc2.suse11/bin/git [.] > blk_SHA1_Block > ESC[31m 5.11%ESC[m git > <prefix>/zlib/local/lib/libz.so.1.2.5 [.] > inflate_fast > ESC[32m 2.61%ESC[m git > <prefix>/zlib/local/lib/libz.so.1.2.5 [.] > adler32 > ESC[32m 1.98%ESC[m git /lib64/libc-2.11.1.so > [.] _int_malloc > ESC[32m 0.86%ESC[m git [kernel] > [k] clear_page_c > > Does this help? Machine has 396GB of RAM if it matters. > > On Thu, Nov 8, 2012 at 4:33 PM, Jeff King <p...@peff.net> wrote: >> On Thu, Nov 08, 2012 at 04:16:59PM -0600, Uri Moszkowicz wrote: >> >>> I ran "git cat-file commit some-tag" for every tag. They seem to be >>> roughly uniformly distributed between 0s and 2s and about 2/3 of the >>> time seems to be system. My disk is mounted over NFS so I tried on the >>> local disk and it didn't make a difference. >>> >>> I have only one 1.97GB pack. I ran "git gc --aggressive" before. >> >> Ah. NFS. That is almost certainly the source of the problem. Git will >> aggressively mmap. I would not be surprised to find that RHEL4's NFS >> implementation is not particularly fast at mmap-ing 2G files, and is >> spending a bunch of time in the kernel servicing the requests. >> >> Aside from upgrading your OS or getting off of NFS, I don't have a lot >> of advice. The performance characteristics you are seeing are so >> grossly off of what is normal that using git is probably going to be >> painful. Your 2s cat-files should be more like .002s. I don't think >> there's anything for git to fix here. >> >> You could try building with NO_MMAP, which will emulate it with pread. >> That might fare better under your NFS implementation. Or it might be >> just as bad. >> >> -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
