[PATCH v2 1/1] sha1_file: Add support for downloading blobs on demand

tiple request within a single git command. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- Documentation/technical/read-object-protocol.txt | 102 cache.h | 1 + config.c | 5 + contri

Re: [RFC PATCH 0/3] Partial clone: promised blobs (formerly "missing blobs")

On 7/16/2017 11:23 AM, Philip Oakley wrote: From: "Jonathan Tan" Sent: Tuesday, July 11, 2017 8:48 PM These patches are part of a set of patches implementing partial clone, as you can see here: https://github.com/jonathantanmy/git/tree/partialclone In that branch,

Re: [PATCH v2 1/1] sha1_file: Add support for downloading blobs on demand

interface for retrieving objects. [1] https://public-inbox.org/git/34efd9e9936fdab331655f5a33a098a72dc134f4.1499800530.git.jonathanta...@google.com/ [2] https://public-inbox.org/git/20170713123951.5cab1...@twelve2.svl.corp.google.com/ On Fri, 14 Jul 2017 09:26:51 -0400 Ben Peart <pe

Re: [RFC/PATCH v4 30/49] odb-helper: add read_object_process()

On 6/20/2017 3:55 AM, Christian Couder wrote: From: Ben Peart <benpe...@microsoft.com> Signed-off-by: Ben Peart <benpe...@microsoft.com> Signed-off-by: Christian Couder <chrisc...@tuxfamily.org> --- odb-helper.c | 202 +++

Re: [PATCH] strbuf: use designated initializers in STRBUF_INIT

On 7/10/2017 12:04 PM, Jeff King wrote: On Mon, Jul 10, 2017 at 10:57:57AM -0400, Ben Peart wrote: If this patch can survive a few releases without complaint, then we can feel more confident that designated initializers are widely supported by our user base. It also is an indication

Re: [RFC/PATCH v4 00/49] Add initial experimental external ODB support

On 7/1/2017 3:41 PM, Christian Couder wrote: On Fri, Jun 23, 2017 at 8:24 PM, Ben Peart <peart...@gmail.com> wrote: On 6/20/2017 3:54 AM, Christian Couder wrote: To be able to better handle some kind of objects, for example big blobs, it would be nice if Git could store its o

Re: [RFC/WIP PATCH] object store classification

On 7/6/2017 10:33 PM, Junio C Hamano wrote: Stefan Beller writes: Subject: Re: [RFC/WIP PATCH] object store classification I thought you are writing different object-store backends and classifying them into many categories (e.g. local, networked, telepathic, etc.) It

Re: [RFC PATCH 3/3] sha1_file: add promised blob hook support

On 7/11/2017 3:48 PM, Jonathan Tan wrote: Teach sha1_file to invoke a hook whenever a blob is requested and unavailable but is promised. The hook is a shell command that can be configured through "git config"; this hook takes in a list of hashes and writes (if successful) the corresponding

Re: [PATCH v5 0/7] Fast git status via a file system watcher

On 7/10/2017 9:36 AM, Ben Peart wrote: On 6/28/2017 1:11 AM, Christian Couder wrote: On Sat, Jun 10, 2017 at 3:40 PM, Ben Peart <peart...@gmail.com> wrote: Changes from V4 include: ... I took a look at this patch series except the last patch ([PATCH v5 7/7] fsmonitor: add a perfo

Re: [PATCH v5 0/7] Fast git status via a file system watcher

On 6/28/2017 1:11 AM, Christian Couder wrote: On Sat, Jun 10, 2017 at 3:40 PM, Ben Peart <peart...@gmail.com> wrote: Changes from V4 include: ... I took a look at this patch series except the last patch ([PATCH v5 7/7] fsmonitor: add a performance test) as Junio reviewed it already, a

Re: [PATCH] strbuf: use designated initializers in STRBUF_INIT

On 7/10/2017 3:03 AM, Jeff King wrote: On Sun, Jul 09, 2017 at 10:05:49AM -0700, Junio C Hamano wrote: René Scharfe writes: I wonder when we can begin to target C99 in git's source, though. :) Let's get the ball rolling by starting to use some of the useful features like

Re: [RFC PATCH v2 1/4] object: remove "used" field from struct object

On 7/19/2017 8:55 PM, Jonathan Tan wrote: On Wed, 19 Jul 2017 17:36:39 -0700 Stefan Beller wrote: On Wed, Jul 19, 2017 at 5:21 PM, Jonathan Tan wrote: The "used" field in struct object is only used by builtin/fsck. Remove that field and modify

Re: [PATCH] sub-process: refactor handshake to common function

On 7/24/2017 5:38 PM, Jonathan Tan wrote: Refactor, into a common function, the version and capability negotiation done when invoking a long-running process as a clean or smudge filter. This will be useful for other Git code that needs to interact similarly with a long-running process.

Re: [RFC PATCH v2 2/4] promised-object, fsck: introduce promised objects

On 7/21/2017 4:33 PM, Jonathan Tan wrote: On Fri, 21 Jul 2017 12:24:52 -0400 Ben Peart <peart...@gmail.com> wrote: Today we have 3.5 million objects * 30 bytes per entry = 105 MB of promises. Given the average developer only hydrates 56K files (2 MB promises) that is 103 MB to do

Re: [RFC PATCH v2 2/4] promised-object, fsck: introduce promised objects

On 7/19/2017 8:21 PM, Jonathan Tan wrote: Currently, Git does not support repos with very large numbers of objects or repos that wish to minimize manipulation of certain blobs (for example, because they are very large) very well, even if the user operates mostly on part of the repo, because

Re: [RFC PATCH v2 4/4] sha1_file: support promised object hook

s that only have a few missing objects (for example, repos that only want to exclude large blobs), and might be tolerable in the future if we have batching support for the most commonly used commands, but is not tolerable now for repos that exclude a large amount of objects. H

Re: [RFC PATCH v2 4/4] sha1_file: support promised object hook

On 7/20/2017 5:18 PM, Jonathan Tan wrote: On Thu, 20 Jul 2017 16:58:16 -0400 Ben Peart <peart...@gmail.com> wrote: This is meant as a temporary measure to ensure that all Git commands work in such a situation. Future patches will update some commands to either tolerate promised o

Re: [RFC PATCH v2 2/4] promised-object, fsck: introduce promised objects

On 7/20/2017 5:13 PM, Jonathan Tan wrote: On Thu, 20 Jul 2017 15:58:51 -0400 Ben Peart <peart...@gmail.com> wrote: On 7/19/2017 8:21 PM, Jonathan Tan wrote: Currently, Git does not support repos with very large numbers of objects or repos that wish to minimize manipulation of certain

Re: [PATCH v5 3/7] fsmonitor: teach git to optionally utilize a file system monitor to speed up detecting new or changed files.

On 6/27/2017 11:43 AM, Christian Couder wrote: On Sat, Jun 10, 2017 at 3:40 PM, Ben Peart <peart...@gmail.com> wrote: +int read_fsmonitor_extension(struct index_state *istate, const void *data, + unsigned long sz) +{ + const char *index = data; + uint32_t hdr_v

Re: [PATCH v6 3/8] convert: Split start_multi_file_filter into two separate functions

On 4/24/2017 12:31 AM, Junio C Hamano wrote: Ben Peart <peart...@gmail.com> writes: Subject: [PATCH v6 3/8] convert: Split start_multi_file_filter into two separate functions Two minor nits, because the capital after ":" looks ugly in shortlog output, and having () there

Re: [PATCH v6 1/8] pkt-line: add packet_read_line_gently()

On 4/24/2017 12:21 AM, Junio C Hamano wrote: Ben Peart <peart...@gmail.com> writes: Add packet_read_line_gently() to enable reading a line without dying on EOF. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- pkt-line.c | 14 +- pkt-line.h | 10

Re: [PATCH v6 1/8] pkt-line: add packet_read_line_gently()

Sorry if you get this twice, somehow Thunderbird converted my response to HTML On 4/24/2017 10:19 PM, Junio C Hamano wrote: Ben Peart <peart...@gmail.com> writes: On 4/24/2017 12:21 AM, Junio C Hamano wrote: Ben Peart <peart...@gmail.com> writes: +{ + int len = pa

Re: Partial clone design (with connectivity check for locally-created objects)

On 8/7/2017 3:21 PM, Jonathan Nieder wrote: Hi, Ben Peart wrote: On Fri, 04 Aug 2017 15:51:08 -0700 Junio C Hamano <gits...@pobox.com> wrote: Jonathan Tan <jonathanta...@google.com> writes: "Imported" objects must be in a packfile that has a ".remote" f

Re: Partial clone design (with connectivity check for locally-created objects)

On 8/7/2017 3:41 PM, Junio C Hamano wrote: Ben Peart <peart...@gmail.com> writes: My concern with this proposal is the combination of 1) writing a new pack file for every git command that ends up bringing down a missing object and 2) gc not compressing those pack files into a singl

Re: [PATCH v2 0/5] Fsck for lazy objects, and (now) actual invocation of loader

On 7/31/2017 5:02 PM, Jonathan Tan wrote: Besides review changes, this patch set now includes my rewritten lazy-loading sha1_file patch, so you can now do this (excerpted from one of the tests): test_create_repo server test_commit -C server 1 1.t abcdefgh HASH=$(git hash-object

Re: [PATCH v2 5/5] sha1_file: support loading lazy objects

On 7/31/2017 5:02 PM, Jonathan Tan wrote: Teach sha1_file to invoke the command configured in extensions.lazyObject whenever an object is requested and unavailable. The usage of the hook can be suppressed through a flag when invoking has_object_file_with_flags() and other similar functions.

Re: [PATCH for NEXT v3 2/2] sub-process: refactor handshake to common function

On 8/7/2017 2:17 PM, Jonathan Tan wrote: On Mon, 7 Aug 2017 19:51:04 +0200 Lars Schneider wrote: On 07 Aug 2017, at 19:21, Jonathan Tan wrote: On Sun, 6 Aug 2017 21:58:24 +0200 Lars Schneider wrote: +

Re: Partial clone design (with connectivity check for locally-created objects)

On 8/4/2017 8:21 PM, Jonathan Tan wrote: On Fri, 04 Aug 2017 15:51:08 -0700 Junio C Hamano wrote: Jonathan Tan writes: "Imported" objects must be in a packfile that has a ".remote" file with arbitrary text (similar to the ".keep" file). They

Re: [RFC PATCH] Updated "imported object" design

On 8/16/2017 5:35 PM, Jonathan Tan wrote: On Wed, 16 Aug 2017 13:32:23 -0700 Junio C Hamano wrote: Jonathan Tan writes: Also, let me know if there's a better way to send out these patches for review. Some of the code here has been reviewed

Re: [RFC PATCH] Updated "imported object" design

On 8/17/2017 5:39 PM, Jonathan Tan wrote: Thanks for your comments. I'll reply to both your e-mails in this one e-mail. This illustrates another place we need to resolve the naming/vocabulary. We should at least be consistent to make it easier to discuss/explain. We obviously went with

[PATCH v1 3/5] fsmonitor: add test cases for fsmonitor extension

the extension and that we get the performance benefits desired. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- t/t7519-status-fsmonitor.sh | 134 1 file changed, 134 insertions(+) create mode 100644 t/t7519-status-fsmonitor.sh diff --git a/t

[PATCH v1 5/5] Add a sample query-fsmonitor hook script that integrates with the cross platform Watchman file watching service.

to use the extension ('git config core.fsmonitor true') and optionally turn on the untracked cache for optimal performance ('git config core.untrackedcache true'). Signed-off-by: Ben Peart <benpe...@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schinde...@gmx.de> --- tem

[PATCH v1 2/5] Teach git to optionally utilize a file system monitor to speed up detecting new or changed files.

dirty is not checked as it cannot have any changes. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- Makefile | 1 + builtin/update-index.c | 1 + cache.h| 5 ++ config.c | 5 ++ dir.c | 13 +++ dir.h

[PATCH v1 0/5] Fast git status via a file system watcher

Credits ~~~ Idea taken and code refactored from http://public-inbox.org/git/1466914464-10358-1-git-send-email-nova...@novalis.org/ Current version as a fork of GFW on GitHub here: https://github.com/benpeart/git-for-windows/tree/fsmonitor Ben Peart (5): dir: make lookup_untracked()

[PATCH v1 4/5] Add documentation for the fsmonitor extension. This includes the core.fsmonitor setting, the query-fsmonitor hook, and the fsmonitor index extension.

Signed-off-by: Ben Peart <benpe...@microsoft.com> --- Documentation/config.txt | 7 +++ Documentation/githooks.txt | 23 +++ Documentation/technical/index-format.txt | 18 ++ 3 files changed, 48 insertions(+) diff

[PATCH v1 1/5] dir: make lookup_untracked() available outside of dir.c

Remove the static qualifier from lookup_untracked() and make it available to other modules by exporting it from dir.h. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- dir.c | 2 +- dir.h | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/dir.c b/dir.c index f451

Re: [PATCH v1 5/5] Add a sample query-fsmonitor hook script that integrates with the cross platform Watchman file watching service.

On 5/15/2017 3:50 PM, David Turner wrote: -Original Message- From: Ben Peart [mailto:peart...@gmail.com] Sent: Monday, May 15, 2017 3:14 PM To: git@vger.kernel.org Cc: gits...@pobox.com; benpe...@microsoft.com; pclo...@gmail.com; johannes.schinde...@gmx.de; David Turner <david.

Re: [PATCH v1 2/5] Teach git to optionally utilize a file system monitor to speed up detecting new or changed files.

CE_FSMONITOR_DIRTY to ensure the file is checked. This is pretty simple to demonstrate - a simple "git reset HEAD~1" will do it as a mixed reset updates the index but doesn't touch the files in the working directory. On 05/15/2017 12:13 PM, Ben Peart wrote: diff --git a/cache.h b/cac

Re: [PATCH v1 2/5] Teach git to optionally utilize a file system monitor to speed up detecting new or changed files.

On 5/15/2017 9:55 PM, Ben Peart wrote: On 5/15/2017 8:34 PM, Jeff King wrote: On Tue, May 16, 2017 at 12:22:14AM +, brian m. carlson wrote: On Mon, May 15, 2017 at 03:13:44PM -0400, Ben Peart wrote: +istate->last_update = (time_t)ntohll(*(uint64_t *)index); +index += siz

Re: [PATCH v1 2/5] Teach git to optionally utilize a file system monitor to speed up detecting new or changed files.

On 5/16/2017 3:13 PM, Johannes Sixt wrote: Am 16.05.2017 um 19:17 schrieb Ben Peart: OK, now I'm confused as to the best path for adding a get_be64. This one is trivial: #define get_be64(p)ntohll(*(uint64_t *)(p)) I cringe when I see a cast like this. Unless you can guarantee that p

Re: [PATCH v1 2/5] Teach git to optionally utilize a file system monitor to speed up detecting new or changed files.

On 5/15/2017 8:34 PM, Jeff King wrote: On Tue, May 16, 2017 at 12:22:14AM +, brian m. carlson wrote: On Mon, May 15, 2017 at 03:13:44PM -0400, Ben Peart wrote: + istate->last_update = (time_t)ntohll(*(uint64_t *)index); + index += sizeof(uint64_t); + + ewah_s

Re: [PATCH v1 2/5] Teach git to optionally utilize a file system monitor to speed up detecting new or changed files.

On 5/15/2017 5:21 PM, David Turner wrote: -Original Message- From: Ben Peart [mailto:peart...@gmail.com] Sent: Monday, May 15, 2017 3:14 PM To: git@vger.kernel.org Cc: gits...@pobox.com; benpe...@microsoft.com; pclo...@gmail.com; johannes.schinde...@gmx.de; David Turner <david.

Re: [PATCH 0/1] Preserve the untracked cache across checkout, reset --hard, etc

On 5/9/2017 1:02 AM, Junio C Hamano wrote: David Turner writes: Can you actually keep the email address as my Twopensource one? I want to make sure that Twitter, my employer at the time, gets credit for this work (just as I want to make sure that my current

Re: [RFC/PATCH v4 09/49] Add initial external odb support

On 6/20/2017 3:54 AM, Christian Couder wrote: From: Jeff King Signed-off-by: Christian Couder I'd suggest you make the function names consistent with the capabilities flags (ie get, put, have) both here in odb_helper.c/h and in external_odb.c/h.

Re: [RFC/PATCH v4 00/49] Add initial experimental external ODB support

...@google.com/ [4] https://public-inbox.org/git/20170602232508.ga21...@aiede.mtv.corp.google.com/ * External object database This RFC patch series shows in the tests: - how to use another git repository as an external ODB (storing Git objects) - how to use an http server as an exte

Re: [RFC/PATCH v4 07/49] Git/Packet.pm: add packet_initialize()

I like where this ends but it seems to me that patches 6, 7 and 8 should just get merged into patch 4 and 5. On 6/20/2017 3:54 AM, Christian Couder wrote: Add a function to initialize the communication. And use this function in 't/t0021/rot13-filter.pl'. Signed-off-by: Christian Couder

Re: [PATCH 0/1] Preserve the untracked cache across checkout, reset --hard, etc

On 5/9/2017 8:51 AM, Ben Peart wrote: On 5/9/2017 1:02 AM, Junio C Hamano wrote: David Turner <david.tur...@twosigma.com> writes: Can you actually keep the email address as my Twopensource one? I want to make sure that Twitter, my employer at the time, gets credit for this work (jus

Re: [PATCH v2 6/6] fsmonitor: add a sample query-fsmonitor hook script for Watchman

On 5/26/2017 5:47 AM, Ævar Arnfjörð Bjarmason wrote: On Wed, May 24, 2017 at 3:12 PM, Christian Couder <christian.cou...@gmail.com> wrote: On Thu, May 18, 2017 at 10:13 PM, Ben Peart <peart...@gmail.com> wrote: This hook script integrates the new fsmonitor capabilities of git wi

Re: [PATCH v2 0/6] Fast git status via a file system watcher

On 5/24/2017 6:54 AM, Christian Couder wrote: Design ~~ A new git hook (query-fsmonitor) must exist and be enabled (core.fsmonitor=true) that takes a time_t formatted as a string and outputs to stdout all files that have been modified since the requested time. Is there a reason why

Re: [PATCH v2 5/6] fsmonitor: add documentation for the fsmonitor extension.

On 5/22/2017 1:28 PM, Ævar Arnfjörð Bjarmason wrote: On Mon, May 22, 2017 at 6:18 PM, Ben Peart <peart...@gmail.com> wrote: On 5/20/2017 8:10 AM, Ævar Arnfjörð Bjarmason wrote: +== File System Monitor cache + + The file system monitor cache tracks files for which the query-fsm

Re: [PATCH v2 3/6] fsmonitor: teach git to optionally utilize a file system monitor to speed up detecting new or changed files.

inning of line */ On 5/18/2017 4:13 PM, Ben Peart wrote: When the index is read from disk, the query-fsmonitor index extension is used to flag the last known potentially dirty index and untracked cach entries. If git finds out some entries are 'fsmonitor-dirty', but are really unchanged (e.g.

[PATCH v2 3/6] fsmonitor: teach git to optionally utilize a file system monitor to speed up detecting new or changed files.

dirty is not checked as it cannot have any changes. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- Makefile | 1 + builtin/update-index.c | 1 + cache.h| 5 ++ config.c | 5 ++ dir.c | 13 +++ dir.h

[PATCH v2 4/6] fsmonitor: add test cases for fsmonitor extension

the extension and that we get the performance benefits desired. All test hooks output a marker file that is used to ensure the hook was actually used to generate the test results. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- t/t7519-status-fsmonitor.sh

[PATCH v2 2/6] dir: make lookup_untracked() available outside of dir.c

Remove the static qualifier from lookup_untracked() and make it available to other modules by exporting it from dir.h. This will be used later when we need to find entries to mark 'fsmonitor dirty.' Signed-off-by: Ben Peart <benpe...@microsoft.com> --- dir.c | 2 +- dir.h | 3 +++ 2

[PATCH v2 1/6] bswap: add 64 bit endianness helper get_be64

Add a new get_be64 macro to enable 64 bit endian conversions on memory that may or may not be aligned. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- compat/bswap.h | 4 1 file changed, 4 insertions(+) diff --git a/compat/bswap.h b/compat/bswap.h index d47c003544..f89fe7f4b5

[PATCH v2 5/6] fsmonitor: add documentation for the fsmonitor extension.

This includes the core.fsmonitor setting, the query-fsmonitor hook, and the fsmonitor index extension. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- Documentation/config.txt | 7 +++ Documentation/githooks.txt | 23 +++ Documen

[PATCH v2 6/6] fsmonitor: add a sample query-fsmonitor hook script for Watchman

-by: Ben Peart <benpe...@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schinde...@gmx.de> --- templates/hooks--query-fsmonitor.sample | 27 +++ 1 file changed, 27 insertions(+) create mode 100644 templates/hooks--query-fsmonitor.sample diff --git

[PATCH v2 0/6] Fast git status via a file system watcher

re: https://github.com/benpeart/git-for-windows/tree/fsmonitor Ben Peart (6): bswap: add 64 bit endianness helper get_be64 dir: make lookup_untracked() available outside of dir.c fsmonitor: teach git to optionally utilize a file system monitor to speed up detecting new or changed files. fsmo

Re: [WIP/RFC 00/23] repository object

Glad to see you tackling this. This is definitely a step in the right direction. I realize that it will take a lot of work and that intermediate steps may just be pushing it the global state one level higher but eventually it would be great to see an entire code path global state free! I'm

Re: [PATCH v2 5/6] fsmonitor: add documentation for the fsmonitor extension.

On 5/20/2017 8:10 AM, Ævar Arnfjörð Bjarmason wrote: +== File System Monitor cache + + The file system monitor cache tracks files for which the query-fsmonitor + hook has told us about changes. The signature for this extension is + { 'F', 'S', 'M', 'N' }. + + The extension starts with + +

Re: [PATCH v1 3/5] fsmonitor: add test cases for fsmonitor extension

On 5/16/2017 12:59 AM, Junio C Hamano wrote: Ben Peart <peart...@gmail.com> writes: Add test cases that ensure status results are correct when using the new fsmonitor extension. Test untracked, modified, and new files by ensuring the results are identical to when not using the ext

Re: [PATCH v2 0/6] Fast git status via a file system watcher

On 5/30/2017 4:33 PM, Christian Couder wrote: On Tue, May 30, 2017 at 8:05 PM, Ben Peart <peart...@gmail.com> wrote: On 5/27/2017 2:57 AM, Christian Couder wrote: On Thu, May 25, 2017 at 3:55 PM, Ben Peart <peart...@gmail.com> wrote: On 5/24/2017 6:54 AM, Christian

Re: [PATCH v3 4/6] fsmonitor: add test cases for fsmonitor extension

On 5/30/2017 9:18 AM, Christian Couder wrote: On Thu, May 25, 2017 at 8:36 PM, Ben Peart <peart...@gmail.com> wrote: [...] diff --git a/t/t7519-status-fsmonitor.sh b/t/t7519-status-fsmonitor.sh new file mode 100755 index 00..395db46d55 --- /dev/null +++ b/t/t7519-status-fsmoni

Re: [PATCH v3 4/6] fsmonitor: add test cases for fsmonitor extension

On 5/30/2017 6:37 PM, Junio C Hamano wrote: Ben Peart <peart...@gmail.com> writes: I did a quick search through the existing test scripts and the majority do not link commands together with && when they are in a sub function like this. I find not having them linked toge

Re: [PATCH v4 0/6] Fast git status via a file system watcher

difference (or something else entirely :)). Thanks, Ben On 6/1/2017 5:13 PM, Ævar Arnfjörð Bjarmason wrote: On Thu, Jun 1, 2017 at 10:51 PM, Ævar Arnfjörð Bjarmason <ava...@gmail.com> wrote: On Thu, Jun 1, 2017 at 5:50 PM, Ben Peart <peart...@gmail.com> wrote: Changes fro

Re: [PATCH v3 4/6] fsmonitor: add test cases for fsmonitor extension

On 5/31/2017 12:33 AM, Christian Couder wrote: +test_expect_success 'refresh_index() invalidates fsmonitor cache' ' + git config core.fsmonitor true && + git config core.untrackedcache true && + clean_repo && + git status && + test_path_is_missing marker && +

Re: [WIP/PATCH 7/6] perf: add a performance test for core.fsmonitor

st. There is certainly noise from run to run but on my machine, the minimum time I saw with fsmonitor was in the .25+ range. Where fsmonitor really becomes useful is when status times get over a second and in extreme cases where it gets to a minute or more. On Fri, Jun 2, 2017 at 2:40 AM, B

Re: [PATCH v4 0/6] Fast git status via a file system watcher

On 6/1/2017 3:57 PM, Ævar Arnfjörð Bjarmason wrote: On Thu, Jun 1, 2017 at 5:50 PM, Ben Peart <peart...@gmail.com> wrote: Changes from V3 include: - update test script based on feedback - update template hook proc with better post-processing code and make it executable

Re: RFC: Would a config fetch.retryCount make sense?

On 6/5/2017 8:04 AM, Lars Schneider wrote: On 01 Jun 2017, at 15:33, Ben Peart <peart...@gmail.com> wrote: On 6/1/2017 8:48 AM, Lars Schneider wrote: Hi, we occasionally see "The remote end hung up unexpectedly" (pkt-line.c:265) on our `git fetch` calls (most n

Re: [WIP/PATCH 7/6] perf: add a performance test for core.fsmonitor

its...@pobox.com>; Ben Peart <peart...@gmail.com>; Nguyễn Thái Ngọc Duy <pclo...@gmail.com>; Johannes Schindelin <johannes.schinde...@gmx.de>; David Turner <david.tur...@twosigma.com>; Jeff King <p...@peff.net>; Christian Couder <christian.cou...@gmail.com>;

Re: [WIP/PATCH 7/6] perf: add a performance test for core.fsmonitor

On 6/2/2017 7:06 PM, Ævar Arnfjörð Bjarmason wrote: I don't have time to update the perf test now or dig into it, but most of what you're describing in this mail doesn't at all match with the ad-hoc tests I ran in

Re: [WIP/PATCH 7/6] perf: add a performance test for core.fsmonitor

On 6/7/2017 5:46 PM, Ævar Arnfjörð Bjarmason wrote: On Wed, Jun 7, 2017 at 9:51 PM, Ben Peart <peart...@gmail.com> wrote: On 6/2/2017 7:06 PM, Ævar Arnfjörð Bjarmason wrote: I don't have time to update the perf test now or dig into it, but most of what you're describing in thi

[PATCH v4 5/6] fsmonitor: add documentation for the fsmonitor extension.

This includes the core.fsmonitor setting, the query-fsmonitor hook, and the fsmonitor index extension. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- Documentation/config.txt | 7 +++ Documentation/githooks.txt | 23 +++ Documen

[PATCH v4 4/6] fsmonitor: add test cases for fsmonitor extension

the extension and that we get the performance benefits desired. All test hooks output a marker file that is used to ensure the hook was actually used to generate the test results. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- t/t7519-status-fsmonitor.sh

[PATCH v4 3/6] fsmonitor: teach git to optionally utilize a file system monitor to speed up detecting new or changed files.

dirty is not checked as it cannot have any changes. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- Makefile | 1 + builtin/update-index.c | 1 + cache.h| 5 ++ config.c | 5 ++ dir.c | 14 +++ dir.h

[PATCH v4 2/6] dir: make lookup_untracked() available outside of dir.c

Remove the static qualifier from lookup_untracked() and make it available to other modules by exporting it from dir.h. This will be used later when we need to find entries to mark 'fsmonitor dirty.' Signed-off-by: Ben Peart <benpe...@microsoft.com> --- dir.c | 2 +- dir.h | 3 +++ 2

[PATCH v4 6/6] fsmonitor: add a sample query-fsmonitor hook script for Watchman

-by: Ben Peart <benpe...@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schinde...@gmx.de> Signed-off-by: ?var Arnfj?r? Bjarmason <ava...@gmail.com> --- templates/hooks--query-fsmonitor.sample | 60 + 1 file changed, 60 insertions(+)

[PATCH v4 0/6] Fast git status via a file system watcher

Changes from V3 include: - update test script based on feedback - update template hook proc with better post-processing code and make it executable Ben Peart (6): bswap: add 64 bit endianness helper get_be64 dir: make lookup_untracked() available outside of dir.c fsmonitor: teach git

[PATCH v4 1/6] bswap: add 64 bit endianness helper get_be64

Add a new get_be64 macro to enable 64 bit endian conversions on memory that may or may not be aligned. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- compat/bswap.h | 4 1 file changed, 4 insertions(+) diff --git a/compat/bswap.h b/compat/bswap.h index d47c003544..f89fe7f4b5

Re: [PATCH] sub-process: fix comment about api-sub-process.txt

On 6/14/2017 2:26 PM, Jonathan Nieder wrote: Christian Couder wrote: Subject: sub-process: fix comment about api-sub-process.txt nit: this one-line description doesn't describe what was wrong and is being fixed. I think something like sub-process: correct path to API docs in

Re: [PATCH v5 7/7] fsmonitor: add a performance test

fsmonitor) 3.03(0.01+0.07) 7519.14: status (fsmonitor=true, cold fs cache, warm fsmonitor) 3.24(0.01+0.04) 7519.16: status -uno (fsmonitor=true, cold fs cache) 2.99(0.03+0.03) 7519.18: status -uall (fsmonitor=true, cold fs cache) 35.07(0.03+0.07) On 6/10/2017 9:40 AM, Ben Peart wrote: Add

[PATCH v5 7/7] fsmonitor: add a performance test

Add a test utility (test-drop-caches) that enables dropping the file system cache on Windows. Add a perf test (p7519-fsmonitor.sh) for fsmonitor. Signed-off-by: Ben Peart <benpe...@microsoft.com> Signed-off-by: Ævar Arnfjörð Bjarmason <ava...@gmail.com> --- Makefile

[PATCH v5 6/7] fsmonitor: add a sample query-fsmonitor hook script for Watchman

-by: Ben Peart <benpe...@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schinde...@gmx.de> Signed-off-by: Ævar Arnfjörð Bjarmason <ava...@gmail.com> --- templates/hooks--query-fsmonitor.sample | 76 + 1 file changed, 76 insertions(+)

[PATCH v5 2/7] dir: make lookup_untracked() available outside of dir.c

Remove the static qualifier from lookup_untracked() and make it available to other modules by exporting it from dir.h. This will be used later when we need to find entries to mark 'fsmonitor dirty.' Signed-off-by: Ben Peart <benpe...@microsoft.com> --- dir.c | 2 +- dir.h | 3 +++ 2

[PATCH v5 4/7] fsmonitor: add test cases for fsmonitor extension

the extension and that we get the performance benefits desired. All test hooks output a marker file that is used to ensure the hook was actually used to generate the test results. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- t/t7519-status-fsmonitor.sh

[PATCH v5 3/7] fsmonitor: teach git to optionally utilize a file system monitor to speed up detecting new or changed files.

dirty is not checked as it cannot have any changes. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- Makefile | 1 + builtin/update-index.c | 1 + cache.h| 5 + config.c | 4 + dir.c | 27 +++-- dir.h

[PATCH v5 1/7] bswap: add 64 bit endianness helper get_be64

Add a new get_be64 macro to enable 64 bit endian conversions on memory that may or may not be aligned. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- compat/bswap.h | 4 1 file changed, 4 insertions(+) diff --git a/compat/bswap.h b/compat/bswap.h index d47c003544..f89fe7f4b5

[PATCH v5 5/7] fsmonitor: add documentation for the fsmonitor extension.

This includes the core.fsmonitor setting, the query-fsmonitor hook, and the fsmonitor index extension. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- Documentation/config.txt | 7 +++ Documentation/githooks.txt | 23 +++ Documen

[PATCH v5 0/7] Fast git status via a file system watcher

. test-drop-caches.c: Add perf helper to drop the disk cache on Windows. p7519-fsmonitor.sh: Add perf test for fsmonitor changes Ben Peart (7): bswap: add 64 bit endianness helper get_be64 dir: make lookup_untracked() available outside of dir.c fsmonitor: teach git to optionally

Re: [PATCH v5 7/7] fsmonitor: add a performance test

On 6/12/2017 6:04 PM, Junio C Hamano wrote: Ben Peart <peart...@gmail.com> writes: diff --git a/t/helper/test-drop-caches.c b/t/helper/test-drop-caches.c new file mode 100644 index 00..80830d920b --- /dev/null +++ b/t/helper/test-drop-caches.c @@ -0,0 +1,107 @@ +#include "

[PATCH v7 00/10] refactor the filter process code into a reusable module

rom: Commit edcc85814c ("convert: add filter..process option", 2016-10-16) keeps the external process running and processes all commands Ben Peart (10): convert: remove erroneous tests for errno == EPIPE pkt-line: fix packet_read_line() to handle len < 0 errors

[PATCH v7 03/10] pkt-line: add packet_read_line_gently()

Add packet_read_line_gently() to enable reading a line without dying on EOF. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- pkt-line.c | 12 pkt-line.h | 11 +++ 2 files changed, 23 insertions(+) diff --git a/pkt-line.c b/pkt-line.c index 6f05b1a4a8..7db9

[PATCH v7 01/10] convert: remove erroneous tests for errno == EPIPE

start_multi_file_filter() and apply_multi_file_filter() currently test for errno == EPIPE but treating EPIPE as an error is already happening from one of the packet_write() functions. Signed-off-by: Ben Peart <benpe...@microsoft.com> Found/Fixed-by: Jeff King <p...@peff.net> Ac

[PATCH v7 06/10] convert: Separate generic structures and variables from the filter specific ones

structure that is used to track the negotiated capabilities of the filter. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- convert.c | 35 --- 1 file changed, 20 insertions(+), 15 deletions(-) diff --git a/convert.c b/convert.c index 4e1d018577..5876218347

[PATCH v7 07/10] convert: Update generic functions to only use generic data structures

Update all functions that are going to be moved into a reusable module so that they only work with the reusable data structures. Move code that is specific to the filter out into the filter specific functions. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- convert.

[PATCH v7 09/10] sub-process: move sub-process functions into separate files

Move the sub-proces functions into sub-process.h/c. Add documentation for the new module in Documentation/technical/api-sub-process.txt Signed-off-by: Ben Peart <benpe...@microsoft.com> --- Documentation/technical/api-sub-process.txt | 59 Ma

[PATCH v7 10/10] convert: Update subprocess_read_status to not die on EOF

Enable sub-processes to gracefully handle when the process dies by updating subprocess_read_status to return an error on EOF instead of dying. Update apply_multi_file_filter to take advantage of the revised subprocess_read_status. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- con

[PATCH v7 04/10] convert: move packet_write_line() into pkt-line as packet_writel()

Add packet_writel() which writes multiple lines in a single call and then calls packet_flush_gently(). Update convert.c to use the new packet_writel() function from pkt-line. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- convert.c | 23 ++- pkt-line.

[PATCH v7 08/10] convert: rename reusable sub-process functions

Do a mechanical rename of the functions that will become the reusable sub-process module. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- convert.c | 40 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/convert.c b/convert.c

[PATCH v7 05/10] convert: split start_multi_file_filter() into two separate functions

() is a protocol specific initialization function that will negotiate the multi-file-filter interface version and capabilities. Signed-off-by: Ben Peart <benpe...@microsoft.com> --- convert.c | 58 ++ 1 file changed, 34 insertions(+), 24 deletions(-)

<    1   2   3   4   5   6   7   >