This is a rebase and extension of Duy's work on git index-helper and watchman support.
Patches 1-14 and are more-or-less just Duy's patches, rebased. I fixed up a race condition with signal handling, but otherwise didn't change much. Patch 15 preserves the untracked cache and watchman extensions across checkouts. Patches 16-18 are new index-helper patches that I wrote to improve the general usability of index-helper. And patch 19 is for discussion only -- if the overall concept is correct, I'll rewrite it into the places where it should go. I didn't do it yet in part because I am not sure folks will agree with it and in part because I didn't want to mess too much with Duy's code before re-sending it. I haven't used this code much. I just wrote patches 16-18 today, in fact. It's got fewer tests than I would like, in part because it is somewhat difficult to test since it involves two separate persistent daemons (index-helper, watchman), and the interaction between them. At Twitter, we're still using something based on my earlier watchman patches[1]. We would like to switch to this (especially if this is the version that mainstream git is going towards). But we have other local patches[2], and I haven't fully finished migrating them to 2.8. My informal testing shows that with the untracked cache and watchman index extensions, and git index-helper[3], performance is slightly better than my earlier code. That's not a surprise, since the index-helper eliminates index verification time. [1] https://github.com/dturner-tw/git/tree/dturner/watchman [2] e.g. something like this: https://github.com/dturner-tw/git/tree/dturner/journal [3] In both cases, I'm using a version which has the SSE ref-parsing and hashmap code; these were rejected due to complexity but they provide a benefit for us, so we still use them. David Turner (5): unpack-trees: preserve index extensions index-helper: rewrite pidfile after daemonizing index-helper: process management index-helper: autorun hack: watchman/untracked cache mashup Nguyễn Thái Ngọc Duy (14): trace.c: add GIT_TRACE_PACK_STATS for pack usage statistics read-cache.c: fix constness of verify_hdr() read-cache: allow to keep mmap'd memory after reading index-helper: new daemon for caching index and related stuff trace.c: add GIT_TRACE_INDEX_STATS for index statistics index-helper: add --strict daemonize(): set a flag before exiting the main process index-helper: add --detach index-helper: add Windows support read-cache: add watchman 'WAMA' extension Add watchman support to reduce index refresh cost read-cache: allow index-helper to prepare shm before git reads it index-helper: use watchman to avoid refreshing index with lstat() update-index: enable/disable watchman support .gitignore | 1 + Documentation/git-index-helper.txt | 56 +++++ Documentation/git.txt | 4 + Makefile | 21 ++ builtin/gc.c | 2 +- builtin/update-index.c | 11 + cache.h | 20 +- config.c | 5 + config.mak.uname | 3 + configure.ac | 8 + daemon.c | 2 +- dir.c | 23 +- dir.h | 6 + environment.c | 3 + git-compat-util.h | 1 + git.c | 37 +++- index-helper.c | 437 ++++++++++++++++++++++++++++++++++++ read-cache.c | 441 +++++++++++++++++++++++++++++++++++-- setup.c | 4 +- sha1_file.c | 24 ++ shm.c | 163 ++++++++++++++ shm.h | 23 ++ t/t7063-status-untracked-cache.sh | 23 ++ t/t7900-index-helper.sh | 23 ++ t/test-lib-functions.sh | 4 + trace.c | 16 ++ trace.h | 1 + unpack-trees.c | 1 + watchman-support.c | 134 +++++++++++ watchman-support.h | 7 + 30 files changed, 1481 insertions(+), 23 deletions(-) create mode 100644 Documentation/git-index-helper.txt create mode 100644 index-helper.c create mode 100644 shm.c create mode 100644 shm.h create mode 100755 t/t7900-index-helper.sh create mode 100644 watchman-support.c create mode 100644 watchman-support.h -- 2.4.2.767.g62658d5-twtrsrc -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
