Re: [PATCH v4 05/10] commit-graph: always load commit-graph information

On 4/29/2018 6:14 PM, Jakub Narebski wrote: Derrick Stolee <dsto...@microsoft.com> writes: Most code paths load commits using lookup_commit() and then parse_commit(). And this automatically loads commit graph if needed, thanks to changes in parse_commit_gently(), which parse_commit(

Re: [PATCH 4/9] get_short_oid: sort ambiguous objects by type, then SHA-1

On 5/1/2018 7:27 AM, Ævar Arnfjörð Bjarmason wrote: On Tue, May 01 2018, Derrick Stolee wrote: On 4/30/2018 6:07 PM, Ævar Arnfjörð Bjarmason wrote: Since we show the commit data in the output that's nicely aligned once we sort by object type. The decision to show tags before commits is pretty

Re: [PATCH 4/9] get_short_oid: sort ambiguous objects by type, then SHA-1

On 4/30/2018 6:07 PM, Ævar Arnfjörð Bjarmason wrote: Change the output emitted when an ambiguous object is encountered so that we show tags first, then commits, followed by trees, and finally blobs. Within each type we show objects in hashcmp(). Before this change the objects were only ordered

Re: [PATCH 0/9] get_short_oid UI improvements

ons(-) This is a good series. Please take a look at my suggestion in Patch 4/9, but feel free to keep this series as written. Reviewed-by: Derrick Stolee <dsto...@microsoft.com>

Re: [PATCH v4 10/10] commit-graph.txt: update design document

On 4/30/2018 7:32 PM, Jakub Narebski wrote: Derrick Stolee <dsto...@microsoft.com> writes: We now calculate generation numbers in the commit-graph file and use them in paint_down_to_common(). Expand the section on generation numbers to discuss how the three special generation n

Re: [PATCH v2 06/11] get_short_oid: sort ambiguous objects by type, then SHA-1

On 5/1/2018 9:39 AM, Ævar Arnfjörð Bjarmason wrote: On Tue, May 01 2018, Derrick Stolee wrote: From: Ævar Arnfjörð Bjarmason <ava...@gmail.com> Here is what I mean by sorting during for_each_abbrev(). This seems to work for me, so I don't know what the issue is with this one-pass ap

Re: [PATCH v5 09/11] commit: use generation number in remove_redundant()

On 5/1/2018 8:47 AM, Derrick Stolee wrote: The static remove_redundant() method is used to filter a list of commits by removing those that are reachable from another commit in the list. This is used to remove all possible merge- bases except a maximal, mutually independent set. To determine

Re: branch --contains / tag --merged inconsistency

On 4/27/2018 12:03 PM, SZEDER Gábor wrote: Szia Feri, I'm moving the IRC discussion here, because this might be a bug report in the end. So, kindly try these steps (103 MB free space required): $ git clone https://github.com/ClusterLabs/pacemaker.git && cd pacemaker [...] $ git branch

Re: [PATCH v4 02/10] commit: add generation number to struct commmit

On 4/28/2018 6:35 PM, Jakub Narebski wrote: Derrick Stolee <dsto...@microsoft.com> writes: The generation number of a commit is defined recursively as follows: * If a commit A has no parents, then the generation number of A is one. * If a commit A has parents, then the generation

Re: [PATCH] coccinelle: avoid wrong transformation suggestions from commit.cocci

t. Signed-off-by: SZEDER Gábor <szeder@gmail.com> I applied this locally on 'next' and ran the check. I succeeded with no changes. Thanks! Reviewed-by: Derrick Stolee <dsto...@microsoft.com> --- contrib/coccinelle/commit.cocci | 10 -- 1 file changed, 4 inse

[RFC PATCH 01/18] docs: Multi-Pack Index (MIDX) Design Notes

. If possible, I'd like to remove it and speed up MIDX writes. -- >8 -- Add a technical documentation file describing the design for the multi-pack index (MIDX). Includes current limitations and future work. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/technical/m

[RFC PATCH 00/18] Multi-pack index (MIDX)

g new prefetch packfiles using the following command: git midx --write --update-head --delete-expired --pack-dir= As that release deploys we will gather more specific numbers on the performance improvements and report them in this thread. Derrick Stolee (18): docs: Multi-Pack Index (MIDX

[RFC PATCH 02/18] midx: specify midx file format

e required to ensure collision counts were low. - We need to identify the two lexicographically closest OIDs for fast abbreviations. Binary search allows this. The current solution presents multiple packfiles as if they were packed into a single packfile with one pack-index. Signed-off-by: Derrick St

[RFC PATCH 11/18] midx: teach git-midx to clear midx files

As a way to troubleshoot unforeseen problems with MIDX files, provide a way to delete "midx-head" and the MIDX it references. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-midx.txt | 12 +++- builtin/midx.c | 31 ++

[RFC PATCH 15/18] midx: use midx for approximate object count

The MIDX files contain a complete object count, so we can report the number of objects in the MIDX. The count remains approximate as there may be overlap between the packfiles not referenced by the MIDX. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- packfile.c | 8 ++--

[RFC PATCH 13/18] t5318-midx.h: confirm git actions are stable

Perform some basic read-only operations that load objects and find abbreviations. As this functionality begins to reference MIDX files, ensure the output matches when using MIDX files and when not using them. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- t/t5318-midx.s

[RFC PATCH 03/18] midx: create core.midx config setting

. This can also be a repo- specific setting if the MIDX is misbehaving in only one repo. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/config.txt | 3 +++ cache.h | 1 + config.c | 5 + environment.c| 2 ++ 4 files chang

[RFC PATCH 04/18] midx: write multi-pack indexes for an object list

The write_midx_file() method takes a list of packfiles and indexed objects with offset information and writes according to the format in Documentation/technical/pack-format.txt. The chunks are separated into methods. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Makefile

[RFC PATCH 05/18] midx: create midx builtin with --write mode

IDX files to alternates. The packfiles are expected to be paired with pack-indexes and are otherwise ignored. This simplifies the implementation and also keeps compatibility with older versions of Git (or changing core.midx to false). Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- .gi

[RFC PATCH 07/18] midx: teach midx --write to update midx-head

ed to be incremental and we expect multiple MIDX files at any point. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-midx.txt | 19 ++- builtin/midx.c | 32 ++-- midx.c | 31 +

[RFC PATCH 17/18] sha1_name: use midx for abbreviations

Create unique_in_midx() to mimic behavior of unique_in_pack(). Create find_abbrev_len_for_midx() to mimic behavior of find_abbrev_len_for_pack(). Consume these methods when interacting with abbreviations. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- sha1_name.

[RFC PATCH 14/18] midx: load midx files when loading packs

h use_midx = 0, then unload the MIDX file and reload the packfiles in to the packed_git struct. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- midx.c | 57 +++ midx.h | 6 --

[RFC PATCH 12/18] midx: teach git-midx to delete expired files

As we write new MIDX files, the existing files are probably not needed. Supply the "--delete-expired" flag to remove these files during the "--write" sub- command. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-midx.txt | 4 built

[RFC PATCH 10/18] midx: use existing midx when writing

When writing a new MIDX file, it is faster to use an existing MIDX file to load the object list and pack offsets and to only inspect pack-indexes for packs not already covered by the MIDX file. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- builtin/midx.

[RFC PATCH 09/18] midx: find details of nth object in midx

The MIDX file stores pack offset information for a list of objects. The nth_midxed_object_* methods provide ways to extract this information. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- midx.c | 55 +++ midx.

[RFC PATCH 08/18] midx: teach git-midx to read midx file details

This subcommand is used by t5318-midx.sh to verify the indexed objects are as expected. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-midx.txt | 23 +++- builtin/midx.c | 59 mid

[RFC PATCH 18/18] packfile: use midx for object loads

When looking for a packed object, first check the MIDX for that object. This reduces thrashing in the MRU list of packfiles. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- midx.c | 84 ++ midx.h | 3 +++ pack

[RFC PATCH 06/18] midx: add t5318-midx.sh test script

Test interactions between the midx builtin and other Git operations. Use both a full repo and a bare repo to ensure the pack directory redirection works correctly. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- t/t5318-midx.sh

[RFC PATCH 16/18] midx: nth_midxed_object_oid() and bsearch_midx()

Using a binary search, we can navigate to the position n within a MIDX file where an object appears in the ordered list of objects. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- midx.c | 30 ++ midx.h | 9 + 2 files changed, 39 insertions(+)

Re: merge-base --is-ancestor A B is unreasonably slow with unrelated history B

On 1/9/2018 10:17 AM, Ævar Arnfjörð Bjarmason wrote: This is a pathological case I don't have time to dig into right now: git branch -D orphan; git checkout --orphan orphan && git reset --hard && touch foo && git add foo && git commit -m"foo" && time git

Re: [PATCH v4 0/4] Add --no-ahead-behind to status

On 1/9/2018 8:15 AM, Johannes Schindelin wrote: Hi Peff, On Tue, 9 Jan 2018, Jeff King wrote: On Mon, Jan 08, 2018 at 03:04:20PM -0500, Jeff Hostetler wrote: I was thinking about something similar to the logic we use today about whether to start reporting progress on other long commands.

Re: [RFC PATCH 00/18] Multi-pack index (MIDX)

On 1/7/2018 5:42 PM, Ævar Arnfjörð Bjarmason wrote: On Sun, Jan 07 2018, Derrick Stolee jotted: git log --oneline --raw --parents Num Packs | Before MIDX | After MIDX | Rel % | 1 pack % --+-+++-- 1 | 35.64 s |35.28 s

Re: [RFC PATCH 01/18] docs: Multi-Pack Index (MIDX) Design Notes

On 1/8/2018 2:32 PM, Jonathan Tan wrote: On Sun, 7 Jan 2018 13:14:42 -0500 Derrick Stolee <sto...@gmail.com> wrote: +Design Details +-- + +- The MIDX file refers only to packfiles in the same directory + as the MIDX file. + +- A special file, 'midx-head', stores th

Re: [RFC PATCH 00/18] Multi-pack index (MIDX)

On 1/10/2018 1:25 PM, Martin Fick wrote: On Sunday, January 07, 2018 01:14:41 PM Derrick Stolee wrote: This RFC includes a new way to index the objects in multiple packs using one file, called the multi-pack index (MIDX). ... The main goals of this RFC are: * Determine interest

Re: [RFC PATCH 00/18] Multi-pack index (MIDX)

On 1/8/2018 5:20 AM, Jeff King wrote: On Sun, Jan 07, 2018 at 07:08:54PM -0500, Derrick Stolee wrote: (Not a critique of this, just a (stupid) question) What's the practical use-case for this feature? Since it doesn't help with --abbrev=40 the speedup is all in the part that ensures we don't

Re: [PATCH 03/14] packed-graph: create git-graph builtin

On 1/25/2018 6:01 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: Teach Git the 'graph' builtin that will be used for writing and reading packed graph files. The current implementation is mostly empty, except for a check that the core.graph setting is enabled and a '

Re: [PATCH 04/14] packed-graph: add format document

On 1/25/2018 5:07 PM, Stefan Beller wrote: On Thu, Jan 25, 2018 at 6:02 AM, Derrick Stolee <sto...@gmail.com> wrote: Add document specifying the binary format for packed graphs. This format allows for: * New versions. * New hash functions and hash lengths. * Optional extensions. Basic

Re: [PATCH 01/14] graph: add packed graph design document

On 1/25/2018 4:14 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: Add Documentation/technical/packed-graph.txt with details of the planned packed graph feature, including future plans. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentati

Re: [PATCH 03/14] packed-graph: create git-graph builtin

On 1/25/2018 4:45 PM, Stefan Beller wrote: On Thu, Jan 25, 2018 at 6:02 AM, Derrick Stolee <sto...@gmail.com> wrote: Teach Git the 'graph' builtin that will be used for writing and reading packed graph files. The current implementation is mostly empty, except for a check that the core

Re: [PATCH 06/14] packed-graph: implement git-graph --write

On 1/25/2018 6:28 PM, Stefan Beller wrote: On Thu, Jan 25, 2018 at 6:02 AM, Derrick Stolee <sto...@gmail.com> wrote: + +$ git midx --write midx? Looks like I missed some replacements as I was building this. Now you see how I hope the fe

Re: [PATCH 02/14] packed-graph: add core.graph setting

On 1/25/2018 4:43 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: The packed graph feature is controlled by the new core.graph config setting. This defaults to 0, so the feature is opt-in. The intention of core.graph is that a user can always stop checking for or p

Re: [PATCH 04/14] packed-graph: add format document

On 1/25/2018 5:06 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: Add document specifying the binary format for packed graphs. This format allows for: * New versions. * New hash functions and hash lengths. * Optional extensions. Basic header information is fo

Re: [PATCH 00/14] Serialized Commit Graph

On 1/25/2018 6:06 PM, Ævar Arnfjörð Bjarmason wrote: On Thu, Jan 25 2018, Derrick Stolee jotted: Oops! This is my mistake. The correct command should be: git show-ref -s | git graph --write --update-head --stdin-commits Without "--stdin-commits" the command will walk all pack

Re: [PATCH 01/14] graph: add packed graph design document

On 1/25/2018 3:04 PM, Stefan Beller wrote: On Thu, Jan 25, 2018 at 6:02 AM, Derrick Stolee <sto...@gmail.com> wrote: Add Documentation/technical/packed-graph.txt with details of the planned packed graph feature, including future plans. Signed-off-by: Derrick Stolee <dsto...@micr

[PATCH] packfile: use get_be64() for large offsets

. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- pack-revindex.c | 6 ++ packfile.c | 3 +-- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/pack-revindex.c b/pack-revindex.c index 1b7ebd8d7e..ff5f62c033 100644 --- a/pack-revindex.c +++ b/pack-revindex.c @@ -

Re: [PATCH] sha1_file: remove static strbuf from sha1_file_name()

On 1/16/2018 2:18 AM, Christian Couder wrote: Using a static buffer in sha1_file_name() is error prone and the performance improvements it gives are not needed in most of the callers. So let's get rid of this static buffer and, if necessary or helpful, let's use one in the caller. First: this

Re: [PATCH] describe: use strbuf_add_unique_abbrev() for adding short hashes

On 1/15/2018 12:10 PM, René Scharfe wrote: Call strbuf_add_unique_abbrev() to add an abbreviated hash to a strbuf instead of taking a detour through find_unique_abbrev() and its static buffer. This is shorter and a bit more efficient. Patch generated by Coccinelle (and

Re: [PATCH 02/14] packed-graph: add core.graph setting

On 1/25/2018 3:17 PM, Stefan Beller wrote: On Thu, Jan 25, 2018 at 6:02 AM, Derrick Stolee <sto...@gmail.com> wrote: The packed graph feature is controlled by the new core.graph config setting. This defaults to 0, so the feature is opt-in. The intention of core.graph is that a user can

Re: [PATCH 00/14] Serialized Commit Graph

On 1/25/2018 10:46 AM, Ævar Arnfjörð Bjarmason wrote: On Thu, Jan 25 2018, Derrick Stolee jotted: * 'git log --topo-order -1000' walks all reachable commits to avoid incorrect topological orders, but only needs the commit message for the top 1000 commits. * 'git merge-base ' may walk

[PATCH 00/14] Serialized Commit Graph

request containing the latest version of this patch. P.S. I'm sending this patch from my gmail address to avoid Outlook munging the URLs included in the design document. Derrick Stolee (14): graph: add packed graph design document packed-graph: add core.graph setting packed-graph: create git

[PATCH 04/14] packed-graph: add format document

mit object IDs. * A 256-entry fanout into that list of OIDs. * A list of metadata for the commits. * A list of "large edges" to enable octopus merges. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/technical/graph-format.txt | 88 ++

[PATCH 02/14] packed-graph: add core.graph setting

The packed graph feature is controlled by the new core.graph config setting. This defaults to 0, so the feature is opt-in. The intention of core.graph is that a user can always stop checking for or parsing packed graph files if core.graph=0. Signed-off-by: Derrick Stolee <dsto...@microsoft.

[PATCH 01/14] graph: add packed graph design document

Add Documentation/technical/packed-graph.txt with details of the planned packed graph feature, including future plans. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/technical/packed-graph.txt | 185 +++ 1 file changed, 185 inse

[PATCH 03/14] packed-graph: create git-graph builtin

Teach Git the 'graph' builtin that will be used for writing and reading packed graph files. The current implementation is mostly empty, except for a check that the core.graph setting is enabled and a '--pack-dir' option. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentati

[PATCH 09/14] packed-graph: implement git-graph --clear

Teach Git to delete the current 'graph_head' file and the packed graph it references. This is a good safety valve if somehow the file is corrupted and needs to be recalculated. Since the packed graph is a summary of contents already in the ODB, it can be regenerated. Signed-off-by: Derrick Stolee

[PATCH 08/14] graph: implement git-graph --update-head

It is possible to have multiple packed graph files in a pack directory, but only one is important at a time. Use a 'graph_head' file to point to the important file. Teach git-graph to write 'graph_head' upon writing a new packed graph file. Signed-off-by: Derrick Stolee <dsto...@microsoft.

[PATCH 06/14] packed-graph: implement git-graph --write

Teach git-graph to write graph files. Create new test script to verify this command succeeds without failure. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-graph.txt | 26 ++ builtin/graph.c | 37 ++-- t/t5319-gr

[PATCH 14/14] packed-graph: teach git-graph to read commits

, 700,000+ commits were added to the graph file starting from 'master' in 7-9 seconds, depending on the number of packfiles in the repo (1, 24, or 120). Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- builtin/graph.c | 33 + packed-graph.c

[PATCH 05/14] packed-graph: implement construct_graph()

Teach Git to write a packed graph file by checking all packed objects to see if they are commits, then store the file in the given pack directory. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Makefile | 1 + packed-graph.c

[PATCH 07/14] packed-graph: implement git-graph --read

Teach git-graph to read packed graph files and summarize their contents. Use the --read option to verify the contents of a graph file in the graph tests. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-graph.txt | 7 +++ builtin/graph.c

[PATCH 12/14] packed-graph: read only from specific pack-indexes

Teach git-graph to inspect the objects only in a certain list of pack-indexes within the given pack directory. This allows updating the graph iteratively, since we add all commits stored in a previous packed graph. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentati

[PATCH 13/14] packed-graph: close under reachability

Teach construct_graph() to walk all parents from the commits discovered in packfiles. This prevents gaps given by loose objects or previously-missed packfiles. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- packed-graph.c | 26 ++ t/t5319-graph.s

[PATCH 11/14] commit: integrate packed graph with commit parsing

s | 0.7s | -88% | | branch -vv | 0.42s | 0.27s | -35% | | rev-list --all | 6.4s | 1.0s | -84% | | rev-list --all --objects | 32.6s | 27.6s | -15% | Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- alloc.c |

[PATCH 10/14] packed-graph: teach git-graph --delete-expired

Teach git-graph to delete the graph previously referenced by 'graph_head' when writing a new graph file and updating 'graph_head'. This prevents data creep by storing a list of useless graphs. Be careful to not delete the graph if the file did not change. Signed-off-by: Derrick Stolee <d

[PATCH v2 01/14] commit-graph: add format document

r commit would cause an extra level of indirection for every merge commit. (Octopus merges suffer from this indirection, but they are very rare.) Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/technical/commit-graph-format.txt | 89 + 1 file changed

[PATCH v2 11/14] commit: integrate commit graph with commit parsing

.7s | -88% | | branch -vv | 0.42s | 0.27s | -35% | | rev-list --all | 6.4s | 1.0s | -84% | | rev-list --all --objects | 32.6s | 27.6s | -15% | Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- alloc.c

[PATCH v2 06/14] commit-graph: implement git-commit-graph --read

Teach git-commit-graph to read commit graph files and summarize their contents. Use the --read option to verify the contents of a commit graph file in the tests. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 7 ++ builtin/commit-g

[PATCH v2 08/14] commit-graph: implement git-commit-graph --clear

Teach Git to delete the current 'graph_head' file and the commit graph it references. This is a good safety valve if somehow the file is corrupted and needs to be recalculated. Since the commit graph is a summary of contents already in the ODB, it can be regenerated. Signed-off-by: Derrick Stolee

[PATCH v2 03/14] commit-graph: create git-commit-graph builtin

Teach git the 'commit-graph' builtin that will be used for writing and reading packed graph files. The current implementation is mostly empty, except for a '--pack-dir' option. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- .gitignore | 1 + Documentati

[PATCH v2 00/14] Serialized Git Commit Graph

g in git.git (Jan 2018, #03; Tue, 23) [2] https://github.com/derrickstolee/git/pull/2 A GitHub pull request containing the latest version of this patch. Derrick Stolee (14): commit-graph: add format document graph: add commit graph design document commit-graph: create git-commit-graph

[PATCH v2 10/14] commit-graph: add core.commitgraph setting

The commit graph feature is controlled by the new core.commitgraph config setting. This defaults to 0, so the feature is opt-in. The intention of core.commitgraph is that a user can always stop checking for or parsing commit graph files if core.commitgraph=0. Signed-off-by: Derrick Stolee <d

[PATCH v2 02/14] graph: add commit graph design document

Add Documentation/technical/commit-graph.txt with details of the planned commit graph feature, including future plans. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/technical/commit-graph.txt | 189 +++ 1 file changed, 189 inse

[PATCH v2 12/14] commit-graph: read only from specific pack-indexes

Teach git-commit-graph to inspect the objects only in a certain list of pack-indexes within the given pack directory. This allows updating the commit graph iteratively, since we add all commits stored in a previous commit graph. Signed-off-by: Derrick Stolee <dsto...@microsoft.

[PATCH v2 09/14] commit-graph: teach git-commit-graph --delete-expired

Teach git-commit-graph to delete the graph previously referenced by 'graph_head' when writing a new graph file and updating 'graph_head'. This prevents data creep by storing a list of useless graphs. Be careful to not delete the graph if the file did not change. Signed-off-by: Derrick Stolee

[PATCH v2 05/14] commit-graph: implement git-commit-graph --write

Teach git-commit-graph to write graph files. Create new test script to verify this command succeeds without failure. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 18 +++ builtin/commit-graph.c | 30 t/t5318-

[PATCH v2 07/14] commit-graph: implement git-commit-graph --update-head

It is possible to have multiple commit graph files in a pack directory, but only one is important at a time. Use a 'graph_head' file to point to the important file. Teach git-commit-graph to write 'graph_head' upon writing a new commit graph file. Signed-off-by: Derrick Stolee <d

[PATCH v2 13/14] commit-graph: close under reachability

Teach construct_commit_graph() to walk all parents from the commits discovered in packfiles. This prevents gaps given by loose objects or previously-missed packfiles. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- commit-graph.c | 26 ++ t/t5318-

[PATCH v2 04/14] commit-graph: implement construct_commit_graph()

Teach Git to write a commit graph file by checking all packed objects to see if they are commits, then store the file in the given pack directory. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Makefile | 1 + commit-graph.c

[PATCH v2 14/14] commit-graph: build graph from starting commits

, 700,000+ commits were added to the graph file starting from 'master' in 7-9 seconds, depending on the number of packfiles in the repo (1, 24, or 120). Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 7 ++- builtin/commit-g

Re: [PATCH v2 10/27] protocol: introduce enum protocol_version value protocol_v2

On 1/25/2018 6:58 PM, Brandon Williams wrote: Introduce protocol_v2, a new value for 'enum protocol_version'. Subsequent patches will fill in the implementation of protocol_v2. Signed-off-by: Brandon Williams --- builtin/fetch-pack.c | 3 +++ builtin/receive-pack.c | 6

Re: [PATCH v2 09/27] transport: store protocol version

On 1/25/2018 6:58 PM, Brandon Williams wrote: + switch (data->version) { + case protocol_v1: + case protocol_v0: + refs = fetch_pack(, data->fd, data->conn, + refs_tmp ? refs_tmp : transport->remote_refs, +

Re: [PATCH v2 05/27] upload-pack: factor out processing lines

On 1/26/2018 4:33 PM, Brandon Williams wrote: On 01/26, Stefan Beller wrote: On Thu, Jan 25, 2018 at 3:58 PM, Brandon Williams wrote: Factor out the logic for processing shallow, deepen, deepen_since, and deepen_not lines into their own functions to simplify the

Re: [PATCH v2 08/27] connect: discover protocol version outside of get_remote_heads

On 1/25/2018 6:58 PM, Brandon Williams wrote: In order to prepare for the addition of protocol_v2 push the protocol version discovery outside of 'get_remote_heads()'. This will allow for keeping the logic for processing the reference advertisement for protocol_v1 and protocol_v0 separate from

Re: [PATCH v2 05/14] commit-graph: implement git-commit-graph --write

On 2/2/2018 5:48 PM, Junio C Hamano wrote: Stefan Beller writes: It is true for git-submodule and a few others (the minority of commands IIRC) git-tag for example takes subcommands such as --list or --verify.

Re: [PATCH 0/2] Refactor hash search with fanout table

le you are doing this, I'm guessing you could use your new method to replace (and maybe speed up) the binary search in sha1_name.c:find_abbrev_len_for_pack(). Otherwise, I can take a stab at it next week. Please add Reviewed-by: Derrick Stolee <dsto...@microsoft.com> Thanks, -Stolee

Re: [PATCH v2 12/27] serve: introduce git-serve

On 1/25/2018 6:58 PM, Brandon Williams wrote: Introduce git-serve, the base server for protocol version 2. Protocol version 2 is intended to be a replacement for Git's current wire protocol. The intention is that it will be a simpler, less wasteful protocol which can evolve over time.

Re: [PATCH v2 00/27] protocol version 2

Sorry for chiming in with mostly nitpicks so late since sending this version. Mostly, I tried to read it to see if I could understand the scope of the patch and how this code worked before. It looks very polished, so I the nits were the best I could do. On 1/25/2018 6:58 PM, Brandon Williams

Re: [PATCH v2 14/27] connect: request remote refs using v2

On 1/25/2018 6:58 PM, Brandon Williams wrote: Teach the client to be able to request a remote's refs using protocol v2. This is done by having a client issue a 'ls-refs' request to a v2 server. Signed-off-by: Brandon Williams --- builtin/upload-pack.c | 10 ++--

Re: [PATCH v2 14/27] connect: request remote refs using v2

On 1/31/2018 3:10 PM, Eric Sunshine wrote: On Wed, Jan 31, 2018 at 10:22 AM, Derrick Stolee <sto...@gmail.com> wrote: On 1/25/2018 6:58 PM, Brandon Williams wrote: +static int process_ref_v2(const char *line, struct ref ***list) +{ + int ret = 1; + int i = 0; nit: you

Re: [PATCH v2 10/27] protocol: introduce enum protocol_version value protocol_v2

On 2/2/2018 5:44 PM, Brandon Williams wrote: On 01/31, Derrick Stolee wrote: On 1/25/2018 6:58 PM, Brandon Williams wrote: Introduce protocol_v2, a new value for 'enum protocol_version'. Subsequent patches will fill in the implementation of protocol_v2. Signed-off-by: Brandon Williams <

Re: [PATCH v2 04/14] commit-graph: implement construct_commit_graph()

On 2/2/2018 10:32 AM, SZEDER Gábor wrote: Teach Git to write a commit graph file by checking all packed objects to see if they are commits, then store the file in the given pack directory. I'm afraid that scanning all packed objects is a bit of a roundabout way to approach this. In my git

Re: [PATCH v2 05/14] commit-graph: implement git-commit-graph --write

On 2/1/2018 6:48 PM, SZEDER Gábor wrote: Teach git-commit-graph to write graph files. Create new test script to verify this command succeeds without failure. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 18 +++ builtin/commit-g

[PATCH v3 01/14] commit-graph: add format document

r commit would cause an extra level of indirection for every merge commit. (Octopus merges suffer from this indirection, but they are very rare.) Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/technical/commit-graph-format.txt | 91 + 1 file changed

Re: [PATCH v3 03/14] commit-graph: create git-commit-graph builtin

On 2/8/2018 4:27 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: Teach git the 'commit-graph' builtin that will be used for writing and reading packed graph files. The current implementation is mostly empty, except for a '--pack-dir' option. Why do we want to use

[PATCH v3 09/14] commit-graph: implement --delete-expired

. Otherwise, they may have an open handle on a graph file that will be deleted by the new call. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 11 -- builtin/commit-graph.c | 73 -- t/t5318-commit-gr

Re: [PATCH v3 01/14] commit-graph: add format document

On 2/8/2018 4:21 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: Add document specifying the binary format for commit graphs. This format allows for: * New versions. * New hash functions and hash lengths. It still is unclear, at least to me, why OID and OID

[PATCH v3 03/14] commit-graph: create git-commit-graph builtin

Teach git the 'commit-graph' builtin that will be used for writing and reading packed graph files. The current implementation is mostly empty, except for a '--pack-dir' option. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- .gitignore | 1 + Documentati

[PATCH v3 00/14] Serialized Git Commit Graph

(Jan 2018, #03; Tue, 23) [2] https://github.com/derrickstolee/git/pull/2 A GitHub pull request containing the latest version of this patch. Derrick Stolee (14): commit-graph: add format document graph: add commit graph design document commit-graph: create git-commit-graph builtin

[PATCH v3 07/14] commit-graph: update graph-head during write

It is possible to have multiple commit graph files in a pack directory, but only one is important at a time. Use a 'graph_head' file to point to the important file. Teach git-commit-graph to write 'graph_head' upon writing a new commit graph file. Signed-off-by: Derrick Stolee <d

[PATCH v3 12/14] commit-graph: close under reachability

Teach write_commit_graph() to walk all parents from the commits discovered in packfiles. This prevents gaps given by loose objects or previously-missed packfiles. Also automatically add commits from the existing graph file, if it exists. Signed-off-by: Derrick Stolee <dsto...@microsoft.

<    1   2   3   4   5   6   7   8   9   10   >