[PATCH v3 13/14] commit-graph: read only from specific pack-indexes

Teach git-commit-graph to inspect the objects only in a certain list of pack-indexes within the given pack directory. This allows updating the commit graph iteratively, since we add all commits stored in a previous commit graph. Signed-off-by: Derrick Stolee <dsto...@microsoft.

[PATCH v3 02/14] graph: add commit graph design document

Add Documentation/technical/commit-graph.txt with details of the planned commit graph feature, including future plans. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/technical/commit-graph.txt | 189 +++ 1 file changed, 189 inse

[PATCH v3 06/14] commit-graph: implement 'git-commit-graph read'

Teach git-commit-graph to read commit graph files and summarize their contents. Use the read subcommand to verify the contents of a commit graph file in the tests. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 16 builtin/commit-g

[PATCH v3 14/14] commit-graph: build graph from starting commits

, 700,000+ commits were added to the graph file starting from 'master' in 7-9 seconds, depending on the number of packfiles in the repo (1, 24, or 120). Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 15 ++- builtin/commit-g

[PATCH v3 04/14] commit-graph: implement write_commit_graph()

Teach Git to write a commit graph file by checking all packed objects to see if they are commits, then store the file in the given pack directory. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Makefile | 1 + commit-graph.c

[PATCH v3 05/14] commit-graph: implement 'git-commit-graph write'

Teach git-commit-graph to write graph files. Create new test script to verify this command succeeds without failure. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 39 + builtin/commit-graph.c | 43 +++ t

[PATCH v3 10/14] commit-graph: add core.commitGraph setting

The commit graph feature is controlled by the new core.commitGraph config setting. This defaults to 0, so the feature is opt-in. The intention of core.commitGraph is that a user can always stop checking for or parsing commit graph files if core.commitGraph=0. Signed-off-by: Derrick Stolee <d

[PATCH v3 08/14] commit-graph: implement 'git-commit-graph clear'

Teach Git to delete the current 'graph_head' file and the commit graph it references. This is a good safety valve if somehow the file is corrupted and needs to be recalculated. Since the commit graph is a summary of contents already in the ODB, it can be regenerated. Signed-off-by: Derrick Stolee

[PATCH v3 07/14] commit-graph: update graph-head during write

It is possible to have multiple commit graph files in a pack directory, but only one is important at a time. Use a 'graph_head' file to point to the important file. Teach git-commit-graph to write 'graph_head' upon writing a new commit graph file. Signed-off-by: Derrick Stolee <d

[PATCH v3 12/14] commit-graph: close under reachability

Teach write_commit_graph() to walk all parents from the commits discovered in packfiles. This prevents gaps given by loose objects or previously-missed packfiles. Also automatically add commits from the existing graph file, if it exists. Signed-off-by: Derrick Stolee <dsto...@microsoft.

[PATCH v3 11/14] commit: integrate commit graph with commit parsing

.7s | -88% | | branch -vv | 0.42s | 0.27s | -35% | | rev-list --all | 6.4s | 1.0s | -84% | | rev-list --all --objects | 32.6s | 27.6s | -15% | Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- alloc.c

[PATCH v3 03/14] commit-graph: create git-commit-graph builtin

Teach git the 'commit-graph' builtin that will be used for writing and reading packed graph files. The current implementation is mostly empty, except for a '--pack-dir' option. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- .gitignore | 1 + Documentati

[PATCH v3 00/14] Serialized Git Commit Graph

(Jan 2018, #03; Tue, 23) [2] https://github.com/derrickstolee/git/pull/2 A GitHub pull request containing the latest version of this patch. Derrick Stolee (14): commit-graph: add format document graph: add commit graph design document commit-graph: create git-commit-graph builtin

[PATCH v3 01/14] commit-graph: add format document

r commit would cause an extra level of indirection for every merge commit. (Octopus merges suffer from this indirection, but they are very rare.) Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/technical/commit-graph-format.txt | 91 + 1 file changed

Re: [PATCH v2 04/14] commit-graph: implement construct_commit_graph()

On 2/7/2018 10:08 AM, SZEDER Gábor wrote: On Mon, Feb 5, 2018 at 5:06 PM, Derrick Stolee <sto...@gmail.com> wrote: On 2/2/2018 10:32 AM, SZEDER Gábor wrote: In my git repo, with 9 pack files at the moment, i.e. not that big a repo and not that many pack files: $ time ./git commit

Re: [PATCH v2 05/14] commit-graph: implement git-commit-graph --write

On 2/5/2018 1:48 PM, Junio C Hamano wrote: Jeff King writes: The big advantage of your scheme is that you can update the graph index without repacking. The traditional advice has been that you should always do a full repack during a gc (since it gives the most delta

Re: [PATCH v2 11/14] commit: integrate commit graph with commit parsing

On 2/1/2018 8:51 PM, Jonathan Tan wrote: On Tue, 30 Jan 2018 16:39:40 -0500 Derrick Stolee <sto...@gmail.com> wrote: +/* global storage */ +struct commit_graph *commit_graph = 0; NULL, not 0. +static int bsearch_graph(struct commit_graph *g, struct object_id *oid, uint32_

Re: [PATCH 025/194] object-store: allow prepare_alt_odb to handle arbitrary repositories

On 2/5/2018 8:19 PM, brian m. carlson wrote: On Mon, Feb 05, 2018 at 03:54:46PM -0800, Stefan Beller wrote: @@ -434,12 +433,12 @@ static int link_alt_odb_entry_the_repository(const char *entry, ent = alloc_alt_odb(pathbuf.buf); /* add the alternate entry */ -

Re: [PATCH v2 07/14] commit-graph: implement git-commit-graph --update-head

directory listings to find graph files. Teach git-commit-graph to write 'graph_head' upon writing a new commit graph file. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 34 ++ builtin/commit-graph.c

Re: [PATCH v2 06/14] commit-graph: implement git-commit-graph --read

On 2/1/2018 7:23 PM, Jonathan Tan wrote: On Tue, 30 Jan 2018 16:39:35 -0500 Derrick Stolee <sto...@gmail.com> wrote: Teach git-commit-graph to read commit graph files and summarize their contents. One overall question - is the "read" command meant to be a command used

Re: [PATCH v2 05/14] commit-graph: implement git-commit-graph --write

On 2/1/2018 6:48 PM, SZEDER Gábor wrote: Teach git-commit-graph to write graph files. Create new test script to verify this command succeeds without failure. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 18 +++ builtin/commit-g

Re: [PATCH v2 04/14] commit-graph: implement construct_commit_graph()

On 2/2/2018 10:32 AM, SZEDER Gábor wrote: Teach Git to write a commit graph file by checking all packed objects to see if they are commits, then store the file in the given pack directory. I'm afraid that scanning all packed objects is a bit of a roundabout way to approach this. In my git

Re: [PATCH v2 10/27] protocol: introduce enum protocol_version value protocol_v2

On 2/2/2018 5:44 PM, Brandon Williams wrote: On 01/31, Derrick Stolee wrote: On 1/25/2018 6:58 PM, Brandon Williams wrote: Introduce protocol_v2, a new value for 'enum protocol_version'. Subsequent patches will fill in the implementation of protocol_v2. Signed-off-by: Brandon Williams <

Re: [PATCH 0/2] Refactor hash search with fanout table

le you are doing this, I'm guessing you could use your new method to replace (and maybe speed up) the binary search in sha1_name.c:find_abbrev_len_for_pack(). Otherwise, I can take a stab at it next week. Please add Reviewed-by: Derrick Stolee <dsto...@microsoft.com> Thanks, -Stolee

Re: [PATCH v2 05/14] commit-graph: implement git-commit-graph --write

On 2/2/2018 5:48 PM, Junio C Hamano wrote: Stefan Beller writes: It is true for git-submodule and a few others (the minority of commands IIRC) git-tag for example takes subcommands such as --list or --verify.

Re: [PATCH v2 14/27] connect: request remote refs using v2

On 1/31/2018 3:10 PM, Eric Sunshine wrote: On Wed, Jan 31, 2018 at 10:22 AM, Derrick Stolee <sto...@gmail.com> wrote: On 1/25/2018 6:58 PM, Brandon Williams wrote: +static int process_ref_v2(const char *line, struct ref ***list) +{ + int ret = 1; + int i = 0; nit: you

Re: [PATCH v2 00/27] protocol version 2

Sorry for chiming in with mostly nitpicks so late since sending this version. Mostly, I tried to read it to see if I could understand the scope of the patch and how this code worked before. It looks very polished, so I the nits were the best I could do. On 1/25/2018 6:58 PM, Brandon Williams

Re: [PATCH v2 12/27] serve: introduce git-serve

On 1/25/2018 6:58 PM, Brandon Williams wrote: Introduce git-serve, the base server for protocol version 2. Protocol version 2 is intended to be a replacement for Git's current wire protocol. The intention is that it will be a simpler, less wasteful protocol which can evolve over time.

Re: [PATCH v2 14/27] connect: request remote refs using v2

On 1/25/2018 6:58 PM, Brandon Williams wrote: Teach the client to be able to request a remote's refs using protocol v2. This is done by having a client issue a 'ls-refs' request to a v2 server. Signed-off-by: Brandon Williams --- builtin/upload-pack.c | 10 ++--

Re: [PATCH v2 10/27] protocol: introduce enum protocol_version value protocol_v2

On 1/25/2018 6:58 PM, Brandon Williams wrote: Introduce protocol_v2, a new value for 'enum protocol_version'. Subsequent patches will fill in the implementation of protocol_v2. Signed-off-by: Brandon Williams --- builtin/fetch-pack.c | 3 +++ builtin/receive-pack.c | 6

Re: [PATCH v2 09/27] transport: store protocol version

On 1/25/2018 6:58 PM, Brandon Williams wrote: + switch (data->version) { + case protocol_v1: + case protocol_v0: + refs = fetch_pack(, data->fd, data->conn, + refs_tmp ? refs_tmp : transport->remote_refs, +

Re: [PATCH v2 08/27] connect: discover protocol version outside of get_remote_heads

On 1/25/2018 6:58 PM, Brandon Williams wrote: In order to prepare for the addition of protocol_v2 push the protocol version discovery outside of 'get_remote_heads()'. This will allow for keeping the logic for processing the reference advertisement for protocol_v1 and protocol_v0 separate from

Re: [PATCH v2 05/27] upload-pack: factor out processing lines

On 1/26/2018 4:33 PM, Brandon Williams wrote: On 01/26, Stefan Beller wrote: On Thu, Jan 25, 2018 at 3:58 PM, Brandon Williams wrote: Factor out the logic for processing shallow, deepen, deepen_since, and deepen_not lines into their own functions to simplify the

[PATCH v2 04/14] commit-graph: implement construct_commit_graph()

Teach Git to write a commit graph file by checking all packed objects to see if they are commits, then store the file in the given pack directory. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Makefile | 1 + commit-graph.c

[PATCH v2 10/14] commit-graph: add core.commitgraph setting

The commit graph feature is controlled by the new core.commitgraph config setting. This defaults to 0, so the feature is opt-in. The intention of core.commitgraph is that a user can always stop checking for or parsing commit graph files if core.commitgraph=0. Signed-off-by: Derrick Stolee <d

[PATCH v2 02/14] graph: add commit graph design document

Add Documentation/technical/commit-graph.txt with details of the planned commit graph feature, including future plans. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/technical/commit-graph.txt | 189 +++ 1 file changed, 189 inse

[PATCH v2 12/14] commit-graph: read only from specific pack-indexes

Teach git-commit-graph to inspect the objects only in a certain list of pack-indexes within the given pack directory. This allows updating the commit graph iteratively, since we add all commits stored in a previous commit graph. Signed-off-by: Derrick Stolee <dsto...@microsoft.

[PATCH v2 09/14] commit-graph: teach git-commit-graph --delete-expired

Teach git-commit-graph to delete the graph previously referenced by 'graph_head' when writing a new graph file and updating 'graph_head'. This prevents data creep by storing a list of useless graphs. Be careful to not delete the graph if the file did not change. Signed-off-by: Derrick Stolee

[PATCH v2 05/14] commit-graph: implement git-commit-graph --write

Teach git-commit-graph to write graph files. Create new test script to verify this command succeeds without failure. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 18 +++ builtin/commit-graph.c | 30 t/t5318-

[PATCH v2 07/14] commit-graph: implement git-commit-graph --update-head

It is possible to have multiple commit graph files in a pack directory, but only one is important at a time. Use a 'graph_head' file to point to the important file. Teach git-commit-graph to write 'graph_head' upon writing a new commit graph file. Signed-off-by: Derrick Stolee <d

[PATCH v2 13/14] commit-graph: close under reachability

Teach construct_commit_graph() to walk all parents from the commits discovered in packfiles. This prevents gaps given by loose objects or previously-missed packfiles. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- commit-graph.c | 26 ++ t/t5318-

[PATCH v2 14/14] commit-graph: build graph from starting commits

, 700,000+ commits were added to the graph file starting from 'master' in 7-9 seconds, depending on the number of packfiles in the repo (1, 24, or 120). Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 7 ++- builtin/commit-g

[PATCH v2 06/14] commit-graph: implement git-commit-graph --read

Teach git-commit-graph to read commit graph files and summarize their contents. Use the --read option to verify the contents of a commit graph file in the tests. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 7 ++ builtin/commit-g

[PATCH v2 08/14] commit-graph: implement git-commit-graph --clear

Teach Git to delete the current 'graph_head' file and the commit graph it references. This is a good safety valve if somehow the file is corrupted and needs to be recalculated. Since the commit graph is a summary of contents already in the ODB, it can be regenerated. Signed-off-by: Derrick Stolee

[PATCH v2 03/14] commit-graph: create git-commit-graph builtin

Teach git the 'commit-graph' builtin that will be used for writing and reading packed graph files. The current implementation is mostly empty, except for a '--pack-dir' option. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- .gitignore | 1 + Documentati

[PATCH v2 01/14] commit-graph: add format document

r commit would cause an extra level of indirection for every merge commit. (Octopus merges suffer from this indirection, but they are very rare.) Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/technical/commit-graph-format.txt | 89 + 1 file changed

[PATCH v2 11/14] commit: integrate commit graph with commit parsing

.7s | -88% | | branch -vv | 0.42s | 0.27s | -35% | | rev-list --all | 6.4s | 1.0s | -84% | | rev-list --all --objects | 32.6s | 27.6s | -15% | Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- alloc.c

[PATCH v2 00/14] Serialized Git Commit Graph

g in git.git (Jan 2018, #03; Tue, 23) [2] https://github.com/derrickstolee/git/pull/2 A GitHub pull request containing the latest version of this patch. Derrick Stolee (14): commit-graph: add format document graph: add commit graph design document commit-graph: create git-commit-graph

Re: [PATCH 06/14] packed-graph: implement git-graph --write

On 1/25/2018 6:28 PM, Stefan Beller wrote: On Thu, Jan 25, 2018 at 6:02 AM, Derrick Stolee <sto...@gmail.com> wrote: + +$ git midx --write midx? Looks like I missed some replacements as I was building this. Now you see how I hope the fe

Re: [PATCH 04/14] packed-graph: add format document

On 1/25/2018 5:07 PM, Stefan Beller wrote: On Thu, Jan 25, 2018 at 6:02 AM, Derrick Stolee <sto...@gmail.com> wrote: Add document specifying the binary format for packed graphs. This format allows for: * New versions. * New hash functions and hash lengths. * Optional extensions. Basic

Re: [PATCH 04/14] packed-graph: add format document

On 1/25/2018 5:06 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: Add document specifying the binary format for packed graphs. This format allows for: * New versions. * New hash functions and hash lengths. * Optional extensions. Basic header information is fo

Re: [PATCH 03/14] packed-graph: create git-graph builtin

On 1/25/2018 6:01 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: Teach Git the 'graph' builtin that will be used for writing and reading packed graph files. The current implementation is mostly empty, except for a check that the core.graph setting is enabled and a '

Re: [PATCH 03/14] packed-graph: create git-graph builtin

On 1/25/2018 4:45 PM, Stefan Beller wrote: On Thu, Jan 25, 2018 at 6:02 AM, Derrick Stolee <sto...@gmail.com> wrote: Teach Git the 'graph' builtin that will be used for writing and reading packed graph files. The current implementation is mostly empty, except for a check that the core

Re: [PATCH 02/14] packed-graph: add core.graph setting

On 1/25/2018 4:43 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: The packed graph feature is controlled by the new core.graph config setting. This defaults to 0, so the feature is opt-in. The intention of core.graph is that a user can always stop checking for or p

Re: [PATCH 01/14] graph: add packed graph design document

On 1/25/2018 4:14 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: Add Documentation/technical/packed-graph.txt with details of the planned packed graph feature, including future plans. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentati

Re: [PATCH 01/14] graph: add packed graph design document

On 1/25/2018 3:04 PM, Stefan Beller wrote: On Thu, Jan 25, 2018 at 6:02 AM, Derrick Stolee <sto...@gmail.com> wrote: Add Documentation/technical/packed-graph.txt with details of the planned packed graph feature, including future plans. Signed-off-by: Derrick Stolee <dsto...@micr

Re: [PATCH 00/14] Serialized Commit Graph

On 1/25/2018 6:06 PM, Ævar Arnfjörð Bjarmason wrote: On Thu, Jan 25 2018, Derrick Stolee jotted: Oops! This is my mistake. The correct command should be: git show-ref -s | git graph --write --update-head --stdin-commits Without "--stdin-commits" the command will walk all pack

Re: [PATCH 02/14] packed-graph: add core.graph setting

On 1/25/2018 3:17 PM, Stefan Beller wrote: On Thu, Jan 25, 2018 at 6:02 AM, Derrick Stolee <sto...@gmail.com> wrote: The packed graph feature is controlled by the new core.graph config setting. This defaults to 0, so the feature is opt-in. The intention of core.graph is that a user can

Re: [PATCH 00/14] Serialized Commit Graph

On 1/25/2018 10:46 AM, Ævar Arnfjörð Bjarmason wrote: On Thu, Jan 25 2018, Derrick Stolee jotted: * 'git log --topo-order -1000' walks all reachable commits to avoid incorrect topological orders, but only needs the commit message for the top 1000 commits. * 'git merge-base ' may walk

[PATCH 12/14] packed-graph: read only from specific pack-indexes

Teach git-graph to inspect the objects only in a certain list of pack-indexes within the given pack directory. This allows updating the graph iteratively, since we add all commits stored in a previous packed graph. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentati

[PATCH 13/14] packed-graph: close under reachability

Teach construct_graph() to walk all parents from the commits discovered in packfiles. This prevents gaps given by loose objects or previously-missed packfiles. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- packed-graph.c | 26 ++ t/t5319-graph.s

[PATCH 11/14] commit: integrate packed graph with commit parsing

s | 0.7s | -88% | | branch -vv | 0.42s | 0.27s | -35% | | rev-list --all | 6.4s | 1.0s | -84% | | rev-list --all --objects | 32.6s | 27.6s | -15% | Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- alloc.c |

[PATCH 05/14] packed-graph: implement construct_graph()

Teach Git to write a packed graph file by checking all packed objects to see if they are commits, then store the file in the given pack directory. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Makefile | 1 + packed-graph.c

[PATCH 07/14] packed-graph: implement git-graph --read

Teach git-graph to read packed graph files and summarize their contents. Use the --read option to verify the contents of a graph file in the graph tests. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-graph.txt | 7 +++ builtin/graph.c

[PATCH 10/14] packed-graph: teach git-graph --delete-expired

Teach git-graph to delete the graph previously referenced by 'graph_head' when writing a new graph file and updating 'graph_head'. This prevents data creep by storing a list of useless graphs. Be careful to not delete the graph if the file did not change. Signed-off-by: Derrick Stolee <d

[PATCH 09/14] packed-graph: implement git-graph --clear

Teach Git to delete the current 'graph_head' file and the packed graph it references. This is a good safety valve if somehow the file is corrupted and needs to be recalculated. Since the packed graph is a summary of contents already in the ODB, it can be regenerated. Signed-off-by: Derrick Stolee

[PATCH 08/14] graph: implement git-graph --update-head

It is possible to have multiple packed graph files in a pack directory, but only one is important at a time. Use a 'graph_head' file to point to the important file. Teach git-graph to write 'graph_head' upon writing a new packed graph file. Signed-off-by: Derrick Stolee <dsto...@microsoft.

[PATCH 06/14] packed-graph: implement git-graph --write

Teach git-graph to write graph files. Create new test script to verify this command succeeds without failure. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-graph.txt | 26 ++ builtin/graph.c | 37 ++-- t/t5319-gr

[PATCH 14/14] packed-graph: teach git-graph to read commits

, 700,000+ commits were added to the graph file starting from 'master' in 7-9 seconds, depending on the number of packfiles in the repo (1, 24, or 120). Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- builtin/graph.c | 33 + packed-graph.c

[PATCH 04/14] packed-graph: add format document

mit object IDs. * A 256-entry fanout into that list of OIDs. * A list of metadata for the commits. * A list of "large edges" to enable octopus merges. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/technical/graph-format.txt | 88 ++

[PATCH 02/14] packed-graph: add core.graph setting

The packed graph feature is controlled by the new core.graph config setting. This defaults to 0, so the feature is opt-in. The intention of core.graph is that a user can always stop checking for or parsing packed graph files if core.graph=0. Signed-off-by: Derrick Stolee <dsto...@microsoft.

[PATCH 00/14] Serialized Commit Graph

request containing the latest version of this patch. P.S. I'm sending this patch from my gmail address to avoid Outlook munging the URLs included in the design document. Derrick Stolee (14): graph: add packed graph design document packed-graph: add core.graph setting packed-graph: create git

[PATCH 01/14] graph: add packed graph design document

Add Documentation/technical/packed-graph.txt with details of the planned packed graph feature, including future plans. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/technical/packed-graph.txt | 185 +++ 1 file changed, 185 inse

[PATCH 03/14] packed-graph: create git-graph builtin

Teach Git the 'graph' builtin that will be used for writing and reading packed graph files. The current implementation is mostly empty, except for a check that the core.graph setting is enabled and a '--pack-dir' option. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentati

[PATCH v2] packfile: use get_be64() for large offsets

. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- pack-revindex.c | 6 ++ packfile.c | 3 +-- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/pack-revindex.c b/pack-revindex.c index 1b7ebd8d7e..ff5f62c033 100644 --- a/pack-revindex.c +++ b/pack-revindex.c @@ -

[PATCH] packfile: use get_be64() for large offsets

. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- pack-revindex.c | 6 ++ packfile.c | 3 +-- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/pack-revindex.c b/pack-revindex.c index 1b7ebd8d7e..ff5f62c033 100644 --- a/pack-revindex.c +++ b/pack-revindex.c @@ -

Re: [PATCH] sha1_file: remove static strbuf from sha1_file_name()

On 1/16/2018 2:18 AM, Christian Couder wrote: Using a static buffer in sha1_file_name() is error prone and the performance improvements it gives are not needed in most of the callers. So let's get rid of this static buffer and, if necessary or helpful, let's use one in the caller. First: this

Re: [PATCH] describe: use strbuf_add_unique_abbrev() for adding short hashes

On 1/15/2018 12:10 PM, René Scharfe wrote: Call strbuf_add_unique_abbrev() to add an abbreviated hash to a strbuf instead of taking a detour through find_unique_abbrev() and its static buffer. This is shorter and a bit more efficient. Patch generated by Coccinelle (and

Re: [RFC PATCH 00/18] Multi-pack index (MIDX)

On 1/10/2018 1:25 PM, Martin Fick wrote: On Sunday, January 07, 2018 01:14:41 PM Derrick Stolee wrote: This RFC includes a new way to index the objects in multiple packs using one file, called the multi-pack index (MIDX). ... The main goals of this RFC are: * Determine interest

Re: merge-base --is-ancestor A B is unreasonably slow with unrelated history B

On 1/9/2018 10:17 AM, Ævar Arnfjörð Bjarmason wrote: This is a pathological case I don't have time to dig into right now: git branch -D orphan; git checkout --orphan orphan && git reset --hard && touch foo && git add foo && git commit -m"foo" && time git

Re: [PATCH v4 0/4] Add --no-ahead-behind to status

On 1/9/2018 8:15 AM, Johannes Schindelin wrote: Hi Peff, On Tue, 9 Jan 2018, Jeff King wrote: On Mon, Jan 08, 2018 at 03:04:20PM -0500, Jeff Hostetler wrote: I was thinking about something similar to the logic we use today about whether to start reporting progress on other long commands.

Re: [RFC PATCH 01/18] docs: Multi-Pack Index (MIDX) Design Notes

On 1/8/2018 2:32 PM, Jonathan Tan wrote: On Sun, 7 Jan 2018 13:14:42 -0500 Derrick Stolee <sto...@gmail.com> wrote: +Design Details +-- + +- The MIDX file refers only to packfiles in the same directory + as the MIDX file. + +- A special file, 'midx-head', stores th

Re: [RFC PATCH 00/18] Multi-pack index (MIDX)

On 1/8/2018 5:20 AM, Jeff King wrote: On Sun, Jan 07, 2018 at 07:08:54PM -0500, Derrick Stolee wrote: (Not a critique of this, just a (stupid) question) What's the practical use-case for this feature? Since it doesn't help with --abbrev=40 the speedup is all in the part that ensures we don't

Re: [RFC PATCH 00/18] Multi-pack index (MIDX)

On 1/7/2018 5:42 PM, Ævar Arnfjörð Bjarmason wrote: On Sun, Jan 07 2018, Derrick Stolee jotted: git log --oneline --raw --parents Num Packs | Before MIDX | After MIDX | Rel % | 1 pack % --+-+++-- 1 | 35.64 s |35.28 s

[RFC PATCH 04/18] midx: write multi-pack indexes for an object list

The write_midx_file() method takes a list of packfiles and indexed objects with offset information and writes according to the format in Documentation/technical/pack-format.txt. The chunks are separated into methods. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Makefile

[RFC PATCH 05/18] midx: create midx builtin with --write mode

IDX files to alternates. The packfiles are expected to be paired with pack-indexes and are otherwise ignored. This simplifies the implementation and also keeps compatibility with older versions of Git (or changing core.midx to false). Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- .gi

[RFC PATCH 07/18] midx: teach midx --write to update midx-head

ed to be incremental and we expect multiple MIDX files at any point. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-midx.txt | 19 ++- builtin/midx.c | 32 ++-- midx.c | 31 +

[RFC PATCH 12/18] midx: teach git-midx to delete expired files

As we write new MIDX files, the existing files are probably not needed. Supply the "--delete-expired" flag to remove these files during the "--write" sub- command. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-midx.txt | 4 built

[RFC PATCH 10/18] midx: use existing midx when writing

When writing a new MIDX file, it is faster to use an existing MIDX file to load the object list and pack offsets and to only inspect pack-indexes for packs not already covered by the MIDX file. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- builtin/midx.

[RFC PATCH 09/18] midx: find details of nth object in midx

The MIDX file stores pack offset information for a list of objects. The nth_midxed_object_* methods provide ways to extract this information. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- midx.c | 55 +++ midx.

[RFC PATCH 17/18] sha1_name: use midx for abbreviations

Create unique_in_midx() to mimic behavior of unique_in_pack(). Create find_abbrev_len_for_midx() to mimic behavior of find_abbrev_len_for_pack(). Consume these methods when interacting with abbreviations. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- sha1_name.

[RFC PATCH 08/18] midx: teach git-midx to read midx file details

This subcommand is used by t5318-midx.sh to verify the indexed objects are as expected. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-midx.txt | 23 +++- builtin/midx.c | 59 mid

[RFC PATCH 18/18] packfile: use midx for object loads

When looking for a packed object, first check the MIDX for that object. This reduces thrashing in the MRU list of packfiles. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- midx.c | 84 ++ midx.h | 3 +++ pack

[RFC PATCH 06/18] midx: add t5318-midx.sh test script

Test interactions between the midx builtin and other Git operations. Use both a full repo and a bare repo to ensure the pack directory redirection works correctly. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- t/t5318-midx.sh

[RFC PATCH 16/18] midx: nth_midxed_object_oid() and bsearch_midx()

Using a binary search, we can navigate to the position n within a MIDX file where an object appears in the ordered list of objects. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- midx.c | 30 ++ midx.h | 9 + 2 files changed, 39 insertions(+)

[RFC PATCH 11/18] midx: teach git-midx to clear midx files

As a way to troubleshoot unforeseen problems with MIDX files, provide a way to delete "midx-head" and the MIDX it references. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-midx.txt | 12 +++- builtin/midx.c | 31 ++

[RFC PATCH 15/18] midx: use midx for approximate object count

The MIDX files contain a complete object count, so we can report the number of objects in the MIDX. The count remains approximate as there may be overlap between the packfiles not referenced by the MIDX. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- packfile.c | 8 ++--

[RFC PATCH 13/18] t5318-midx.h: confirm git actions are stable

Perform some basic read-only operations that load objects and find abbreviations. As this functionality begins to reference MIDX files, ensure the output matches when using MIDX files and when not using them. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- t/t5318-midx.s

[RFC PATCH 14/18] midx: load midx files when loading packs

h use_midx = 0, then unload the MIDX file and reload the packfiles in to the packed_git struct. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- midx.c | 57 +++ midx.h | 6 --

[RFC PATCH 03/18] midx: create core.midx config setting

. This can also be a repo- specific setting if the MIDX is misbehaving in only one repo. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/config.txt | 3 +++ cache.h | 1 + config.c | 5 + environment.c| 2 ++ 4 files chang

<    8   9   10   11   12   13   14   >