On Tue, Feb 11, 2014 at 6:11 PM, Duy Nguyen <pclo...@gmail.com> wrote: > > I have no comments about thread safety improvements (well, not yet). > If you have investigated about git performance on chromium > repositories, could you please sum it up? Threading may be an option > to improve performance, but it's probably not the only option.
Well, the painful operations that we use frequently are pack-objects, checkout, status, and blame. Anything on Windows that touches a lot of files is miserable due to the usual file system slowness on Windows, and luafv.sys (the UAC file virtualization driver) seems to make it much worse. With threading turned on, pack-objects on Windows now takes about twice as long as on Linux, which is still more than a 2x improvement over the non-threaded operation. Checkout is really bad on Windows. The blink repository is ~200K files, and a full clean checkout from the index takes about 10 seconds on Linux, and about 3:30 on Windows. I used the Very Sleepy profiler to see where all the time was spent on Windows: 55% of the time was spent in OpenFile, and 25% in CloseFile (both in win32). My immediate goal is to add threading to checkout, so those file system calls can be done in parallel. Enabling the fscache speeds up status quite a bit. I'm optimistic that parallelizing the stat calls will yield a further improvement. Beyond that, it may not be possible to do much more without using a file system watcher daemon, like facebook does with mercurial. (https://code.facebook.com/posts/218678814984400/scaling-mercurial-at-facebook/) Blame is something that chromium and blink developers use heavily, and it is not unusual for a blame invocation on the blink repository to run for 30 seconds. It seems like it should be possible to parallelize blame, but it requires pack file operations to be thread-safe. Stefan -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
