[PATCH v6 18/21] commit-graph: use string-list API for input

Signed-off-by: Derrick Stolee --- builtin/commit-graph.c | 39 +-- commit-graph.c | 15 +++ commit-graph.h | 7 +++ 3 files changed, 23 insertions(+), 38 deletions(-) diff --git a/builtin/commit-graph.c b/builtin/commit

[PATCH v6 06/21] commit-graph: add 'verify' subcommand

state. Do not report any errors. Helped-by: Ramsay Jones Signed-off-by: Derrick Stolee --- Documentation/git-commit-graph.txt | 6 + builtin/commit-graph.c | 39 ++ commit-graph.c | 23 ++ commit-graph.h

[PATCH v6 03/21] commit-graph: parse commit from chosen graph

Before verifying a commit-graph file against the object database, we need to parse all commits from the given commit-graph file. Create parse_commit_in_graph_one() to target a given struct commit_graph. Signed-off-by: Derrick Stolee --- commit-graph.c | 18 +++--- 1 file changed, 15

[PATCH v6 19/21] commit-graph: add '--reachable' option

When writing commit-graph files, it can be convenient to ask for all reachable commits (starting at the ref set) in the resulting file. This is particularly helpful when writing to stdin is complicated, such as a future integration with 'git gc'. Signed-off-by: Derrick Stolee --- Documentation

[PATCH v6 20/21] gc: automatically write commit-graph files

-by: Derrick Stolee --- Documentation/config.txt | 10 +- Documentation/git-gc.txt | 4 builtin/gc.c | 6 ++ t/t5318-commit-graph.sh | 14 ++ 4 files changed, 33 insertions(+), 1 deletion(-) diff --git a/Documentation/config.txt b/Documentation/config.txt

[PATCH v6 15/21] commit-graph: test for corrupted octopus edge

The commit-graph file has an extra chunk to store the parent int-ids for parents beyond the first parent for octopus merges. Our test repo has a single octopus merge that we can manipulate to demonstrate the 'verify' subcommand detects incorrect values in that chunk. Signed-off-by: Derrick Stolee

[PATCH v6 02/21] commit-graph: fix GRAPH_MIN_SIZE

the magic constants. Signed-off-by: Derrick Stolee --- commit-graph.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index b63a1fc85e..f83f6d2373 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -35,10 +35,11 @@ #define GRAPH_LAST_EDGE

[PATCH v6 00/21] Integrate commit-graph into 'fsck' and 'gc'

_OCTOPUS_EDGES)) +GRAPH_BYTE_FOOTER=$(($GRAPH_OCTOPUS_DATA_OFFSET + 4 * $NUM_OCTOPUS_EDGES)) # usage: corrupt_graph_and_verify # Manipulates the commit-graph file at the position --- Derrick Stolee (21): commit-graph: UNLEAK before die() commit-graph: fix GRAPH_MIN_SIZE commit-gra

[PATCH v6 05/21] commit-graph: load a root tree from specific graph

When lazy-loading a tree for a commit, it will be important to select the tree from a specific struct commit_graph. Create a new method that specifies the commit-graph file and use that in get_commit_tree_in_graph(). Signed-off-by: Derrick Stolee --- commit-graph.c | 12 +--- 1 file

Re: [PATCH v5 18/21] commit-graph: use string-list API for input

On 6/6/2018 8:45 AM, Derrick Stolee wrote: On 6/6/2018 8:26 AM, Ævar Arnfjörð Bjarmason wrote: On Wed, Jun 06 2018, Derrick Stolee wrote: On 6/6/2018 8:11 AM, Ævar Arnfjörð Bjarmason wrote: On Wed, Jun 06 2018, Derrick Stolee wrote: Signed-off-by: Derrick Stolee +    string_list_clear

Re: [PATCH 00/23] Multi-pack-index (MIDX)

On 6/7/2018 10:45 AM, Ævar Arnfjörð Bjarmason wrote: On Thu, Jun 07 2018, Derrick Stolee wrote: To test the performance in this situation, I created a script that organizes the Linux repository in a similar fashion. I split the commit history into 50 parts by creating branches on every 10,000

Re: [PATCH 03/23] midx: add midx builtin

On 6/7/2018 1:20 PM, Duy Nguyen wrote: On Thu, Jun 7, 2018 at 4:03 PM, Derrick Stolee wrote: diff --git a/Documentation/git-midx.txt b/Documentation/git-midx.txt new file mode 100644 index 00..2bd886f1a2 --- /dev/null +++ b/Documentation/git-midx.txt @@ -0,0 +1,29 @@ +git-midx(1

Re: [PATCH 03/23] midx: add midx builtin

On 6/11/2018 5:02 PM, Stefan Beller wrote: Hi Derrick, On Thu, Jun 7, 2018 at 7:03 AM Derrick Stolee wrote: This new 'git midx' builtin will be the plumbing access for writing, reading, and checking multi-pack-index (MIDX) files. The initial implementation is a no-op. Let's talk about

Re: [PATCH 03/23] midx: add midx builtin

On 6/18/2018 3:55 PM, Stefan Beller wrote: But as this is plumbing and users should not need to worry about it this is optional, I would think. The design document is also in 'Documentation/technical' instead of just 'Documentation/'. Do we have a pattern of linking to the technical documents?

Re: [PATCH 01/23] midx: add design document

On 6/11/2018 3:04 PM, Stefan Beller wrote: On Thu, Jun 7, 2018 at 7:03 AM Derrick Stolee wrote: Signed-off-by: Derrick Stolee --- Documentation/technical/midx.txt | 109 +++ 1 file changed, 109 insertions(+) create mode 100644 Documentation/technical/midx.txt

Re: [PATCH 02/23] midx: add midx format details to pack-format.txt

On 6/11/2018 3:19 PM, Stefan Beller wrote: Hi Derrick, On Thu, Jun 7, 2018 at 7:03 AM Derrick Stolee wrote: The multi-pack-index (MIDX) feature generalizes the existing pack- index (IDX) feature by indexing objects across multiple pack-files. Describe the basic file format, using a 12-byte

Re: [PATCH v2 05/31] tree: add repository argument to lookup_tree

On 6/13/2018 7:04 PM, Stefan Beller wrote: diff --git a/commit-graph.c b/commit-graph.c index 71125d7cbb6..88a4b0d2a47 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -261,7 +261,7 @@ static int fill_commit_in_graph(struct commit *item, struct commit_graph *g, uin item->graph_pos =

Re: [PATCH 4/3] ewah: adjust callers of ewah_read_mmap()

On 6/14/2018 11:44 PM, Jeff King wrote: The return value of ewah_read_mmap() is now an ssize_t, since we could (in theory) process up to 32GB of data. This would never happen in practice, but a corrupt or malicious .bitmap or index file could convince us to do so. Aside: Why a 32GB limit?

Re: [PATCH 3/3] ewah: drop ewah_serialize_native function

On 6/15/2018 9:56 AM, Ramsay Jones wrote: On 15/06/18 04:32, Jeff King wrote: We don't call this function, and never have. The on-disk bitmap format uses network-byte-order integers, meaning that we cannot use the native-byte-order format written here. Let's drop it in the name of simplicity.

[PATCH 6/8] ewah_bitmap: delete unused 'ewah_or()'

Reported-by: Ramsay Jones Signed-off-by: Derrick Stolee --- ewah/ewah_bitmap.c | 69 -- ewah/ewok.h| 5 2 files changed, 74 deletions(-) diff --git a/ewah/ewah_bitmap.c b/ewah/ewah_bitmap.c index 3fa3edf2a3..017c677f98 100644 --- a/ewah

[PATCH 3/8] ewah_bitmap: delete unused 'ewah_and()'

Reported-by: Ramsay Jones Signed-off-by: Derrick Stolee --- ewah/ewah_bitmap.c | 68 -- ewah/ewok.h| 5 2 files changed, 73 deletions(-) diff --git a/ewah/ewah_bitmap.c b/ewah/ewah_bitmap.c index b9fdda1d3d..5a4d3a6eb6 100644 --- a/ewah

[PATCH 8/8] ewah_io: delete unused 'ewah_serialize_native()'

Signed-off-by: Derrick Stolee --- ewah/ewah_io.c | 26 -- ewah/ewok.h| 1 - 2 files changed, 27 deletions(-) diff --git a/ewah/ewah_io.c b/ewah/ewah_io.c index 86d3f1d02e..dde736bcd9 100644 --- a/ewah/ewah_io.c +++ b/ewah/ewah_io.c @@ -20,32 +20,6 @@ #include

[PATCH 7/8] ewah_io: delete unused 'ewah_serialize()'

Reported-by: Ramsay Jones Signed-off-by: Derrick Stolee --- ewah/ewah_io.c | 10 -- ewah/ewok.h| 1 - 2 files changed, 11 deletions(-) diff --git a/ewah/ewah_io.c b/ewah/ewah_io.c index bed1994551..86d3f1d02e 100644 --- a/ewah/ewah_io.c +++ b/ewah/ewah_io.c @@ -100,16 +100,6

[PATCH 5/8] ewah_bitmap: delete unused 'ewah_not()'

Reported-by: Ramsay Jones Signed-off-by: Derrick Stolee --- ewah/ewah_bitmap.c | 19 --- ewah/ewok.h| 7 --- 2 files changed, 26 deletions(-) diff --git a/ewah/ewah_bitmap.c b/ewah/ewah_bitmap.c index 559adde37c..3fa3edf2a3 100644 --- a/ewah/ewah_bitmap.c +++ b

[PATCH 2/8] ewah/bitmap.c: delete unused 'bitmap_each_bit()'

Reported-by: Ramsay Jones Signed-off-by: Derrick Stolee --- ewah/bitmap.c | 24 1 file changed, 24 deletions(-) diff --git a/ewah/bitmap.c b/ewah/bitmap.c index d61dc6114a..52f1178db4 100644 --- a/ewah/bitmap.c +++ b/ewah/bitmap.c @@ -129,30 +129,6 @@ void

Re: [PATCH 8/8] ewah_io: delete unused 'ewah_serialize_native()'

On 6/15/2018 11:01 AM, Ramsay Jones wrote: On 15/06/18 15:30, Derrick Stolee wrote: Signed-off-by: Derrick Stolee --- ewah/ewah_io.c | 26 -- ewah/ewok.h| 1 - 2 files changed, 27 deletions(-) This duplicates Jeff's patch #3. Thanks. I got a bit hasty as I

[PATCH 1/8] ewah/bitmap.c: delete unused 'bitmap_clear()'

Reported-by: Ramsay Jones Signed-off-by: Derrick Stolee --- ewah/bitmap.c | 8 1 file changed, 8 deletions(-) diff --git a/ewah/bitmap.c b/ewah/bitmap.c index 756bdd050e..d61dc6114a 100644 --- a/ewah/bitmap.c +++ b/ewah/bitmap.c @@ -45,14 +45,6 @@ void bitmap_set(struct bitmap *self

[PATCH 0/8] Delete unused methods in EWAH bitmap

() * ewah_not() * ewah_or() * ewah_serialize() * ewah_serialize_native() We can also delete the unused methods bitmap_clear() and bitmap_each_bit(). Derrick Stolee (8): ewah/bitmap.c: delete unused 'bitmap_clear()' ewah/bitmap.c: delete unused 'bitmap_each_bit()' ewah_bitmap: delete unused

[PATCH 4/8] ewah_bitmap: delete unused 'ewah_and_not()'

Reported-by: Ramsay Jones Signed-off-by: Derrick Stolee --- ewah/ewah_bitmap.c | 73 -- ewah/ewok.h| 5 2 files changed, 78 deletions(-) diff --git a/ewah/ewah_bitmap.c b/ewah/ewah_bitmap.c index 5a4d3a6eb6..559adde37c 100644 --- a/ewah

Re: [PATCH 0/8] Delete unused methods in EWAH bitmap

On 6/15/2018 10:30 AM, Derrick Stolee wrote: The EWAH bitmap code includes several logical operations that are important for a general-purpose bitmap library. However, Git only uses the XOR operation for storing deltas between reachability bitmaps. This means that we can delete the following

Re: [PATCH 1/8] ewah/bitmap.c: delete unused 'bitmap_clear()'

On 6/15/2018 10:46 AM, Ramsay Jones wrote: On 15/06/18 15:30, Derrick Stolee wrote: Reported-by: Ramsay Jones Signed-off-by: Derrick Stolee --- ewah/bitmap.c | 8 1 file changed, 8 deletions(-) diff --git a/ewah/bitmap.c b/ewah/bitmap.c index 756bdd050e..d61dc6114a 100644

Re: [PATCH v2 05/31] tree: add repository argument to lookup_tree

On 6/14/2018 1:55 PM, Derrick Stolee wrote: On 6/13/2018 7:04 PM, Stefan Beller wrote: diff --git a/commit-graph.c b/commit-graph.c index 71125d7cbb6..88a4b0d2a47 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -261,7 +261,7 @@ static int fill_commit_in_graph(struct commit *item, struct

Re: [PATCH v2 0/7] Delete unused methods in EWAH bitmap

On 6/15/2018 2:51 PM, Junio C Hamano wrote: Derrick Stolee writes: The EWAH bitmap code includes several logical operations that are important for a general-purpose bitmap library. However, Git only uses the XOR operation for storing deltas between reachability bitmaps. This means that we can

Re: [PATCH 4/3] ewah: adjust callers of ewah_read_mmap()

On 6/15/2018 1:31 PM, Jeff King wrote: On Fri, Jun 15, 2018 at 09:41:46AM -0700, Junio C Hamano wrote: Derrick Stolee writes: On 6/14/2018 11:44 PM, Jeff King wrote: The return value of ewah_read_mmap() is now an ssize_t, since we could (in theory) process up to 32GB of data. This would

[PATCH v2 7/7] ewah_io: delete unused 'ewah_serialize()'

Reported-by: Ramsay Jones Signed-off-by: Derrick Stolee --- ewah/ewah_io.c | 10 -- ewah/ewok.h| 1 - 2 files changed, 11 deletions(-) diff --git a/ewah/ewah_io.c b/ewah/ewah_io.c index bed1994551..86d3f1d02e 100644 --- a/ewah/ewah_io.c +++ b/ewah/ewah_io.c @@ -100,16 +100,6

[PATCH v2 2/7] ewah/bitmap.c: delete unused 'bitmap_each_bit()'

Reported-by: Ramsay Jones Signed-off-by: Derrick Stolee --- ewah/bitmap.c | 24 ewah/ewok.h | 1 - 2 files changed, 25 deletions(-) diff --git a/ewah/bitmap.c b/ewah/bitmap.c index d61dc6114a..52f1178db4 100644 --- a/ewah/bitmap.c +++ b/ewah/bitmap.c @@ -129,30

[PATCH v2 1/7] ewah/bitmap.c: delete unused 'bitmap_clear()'

Reported-by: Ramsay Jones Signed-off-by: Derrick Stolee --- ewah/bitmap.c | 8 ewah/ewok.h | 1 - 2 files changed, 9 deletions(-) diff --git a/ewah/bitmap.c b/ewah/bitmap.c index 756bdd050e..d61dc6114a 100644 --- a/ewah/bitmap.c +++ b/ewah/bitmap.c @@ -45,14 +45,6 @@ void bitmap_set

[PATCH v2 4/7] ewah_bitmap: delete unused 'ewah_and_not()'

Reported-by: Ramsay Jones Signed-off-by: Derrick Stolee --- ewah/ewah_bitmap.c | 73 -- ewah/ewok.h| 5 2 files changed, 78 deletions(-) diff --git a/ewah/ewah_bitmap.c b/ewah/ewah_bitmap.c index 5a4d3a6eb6..559adde37c 100644 --- a/ewah

[PATCH v2 6/7] ewah_bitmap: delete unused 'ewah_or()'

Reported-by: Ramsay Jones Signed-off-by: Derrick Stolee --- ewah/ewah_bitmap.c | 69 -- ewah/ewok.h| 5 2 files changed, 74 deletions(-) diff --git a/ewah/ewah_bitmap.c b/ewah/ewah_bitmap.c index 3fa3edf2a3..017c677f98 100644 --- a/ewah

[PATCH v2 5/7] ewah_bitmap: delete unused 'ewah_not()'

Reported-by: Ramsay Jones Signed-off-by: Derrick Stolee --- ewah/ewah_bitmap.c | 19 --- ewah/ewok.h| 7 --- 2 files changed, 26 deletions(-) diff --git a/ewah/ewah_bitmap.c b/ewah/ewah_bitmap.c index 559adde37c..3fa3edf2a3 100644 --- a/ewah/ewah_bitmap.c +++ b

[PATCH v2 3/7] ewah_bitmap: delete unused 'ewah_and()'

Reported-by: Ramsay Jones Signed-off-by: Derrick Stolee --- ewah/ewah_bitmap.c | 68 -- ewah/ewok.h| 5 2 files changed, 73 deletions(-) diff --git a/ewah/ewah_bitmap.c b/ewah/ewah_bitmap.c index b9fdda1d3d..5a4d3a6eb6 100644 --- a/ewah

[PATCH v2 0/7] Delete unused methods in EWAH bitmap

() * ewah_not() * ewah_or() * ewah_serialize() We can also delete the unused methods bitmap_clear() and bitmap_each_bit(). Derrick Stolee (7): ewah/bitmap.c: delete unused 'bitmap_clear()' ewah/bitmap.c: delete unused 'bitmap_each_bit()' ewah_bitmap: delete unused 'ewah_and()' ewah_bitmap

Re: [PATCH 05/23] midx: write header information to lockfile

On 6/19/2018 10:59 AM, Duy Nguyen wrote: On Tue, Jun 19, 2018 at 2:54 PM Derrick Stolee wrote: On 6/12/2018 11:00 AM, Duy Nguyen wrote: On Thu, Jun 7, 2018 at 7:01 PM Derrick Stolee wrote: diff --git a/midx.c b/midx.c index 616af66b13..3e55422a21 100644 --- a/midx.c +++ b/midx.c @@ -1,9

Re: [PATCH 01/15] contrib: add cocci script to replace index compat macros

On 6/19/2018 10:51 AM, Duy Nguyen wrote: On Tue, Jun 19, 2018 at 1:41 PM Derrick Stolee wrote: Duy, Here is the patch that was generated by `make coccicheck`. Thanks, -Stolee -->8-- --- builtin/add.c Ah right. This is on purpose. I think I mentioned in the commit message that buil

Re: [PATCH] ewah: delete unused 'rlwit_discharge_empty()'

On 6/19/2018 5:51 PM, Ramsay Jones wrote: From: Junio C Hamano Complete the removal of unused 'ewah bitmap' code by removing the now unused 'rlwit_discharge_empty()' function. Also, the 'ewah_clear()' function can now be made a file-scope static symbol. Signed-off-by: Ramsay Jones --- Hi

Re: [PATCH 03/35] object: add repository argument to lookup_unknown_object

On 6/13/2018 3:30 PM, Stefan Beller wrote: On Wed, Jun 6, 2018 at 12:38 PM Duy Nguyen wrote: On Wed, May 30, 2018 at 2:47 AM, Stefan Beller wrote: diff --git a/object.c b/object.c index 4de4fa58d59..def3c71cac2 100644 --- a/object.c +++ b/object.c @@ -177,7 +177,7 @@ void

Re: [PATCH v2 00/31] object-store: lookup_commit

On 6/13/2018 7:04 PM, Stefan Beller wrote * Once this is in good shape we can talk about converting parts of the revision walking code, This is another reason why I'll be waiting for this series of series. I plan to rework the revision walking code around sort_in_topological_order(), but

Re: [PATCH v2 00/31] object-store: lookup_commit

On 6/13/2018 9:23 PM, Derrick Stolee wrote: On 6/13/2018 7:04 PM, Stefan Beller wrote * Once this is in good shape we can talk about converting parts of the revision    walking code, This is another reason why I'll be waiting for this series of series. I plan to rework the revision

Re: [PATCH 03/35] object: add repository argument to lookup_unknown_object

On 5/29/2018 8:47 PM, Stefan Beller wrote: From: Jonathan Nieder Add a repository argument to allow callers of lookup_unknown_object to be more specific about which repository to handle. This is a small mechanical change; it doesn't change the implementation to handle repositories other than

Re: [PATCH 06/35] blob: add repository argument to lookup_blob

On 5/29/2018 8:47 PM, Stefan Beller wrote: Add a repository argument to allow the callers of lookup_blob to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet. As with

Re: [PATCH v3 06/20] commit-graph: add 'verify' subcommand

On 5/27/2018 6:55 PM, Jakub Narebski wrote: Derrick Stolee writes: If the commit-graph file becomes corrupt, we need a way to verify that its contents match the object database. In the manner of 'git fsck' we will implement a 'git commit-graph verify' subcommand to report all issues

Re: [PATCH 04/35] object: add repository argument to parse_object_buffer

On 5/29/2018 8:47 PM, Stefan Beller wrote: Add a repository argument to allow the callers of parse_object_buffer to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet. As

Re: [PATCH v3 09/20] commit-graph: verify corrupt OID fanout and lookup

On 5/30/2018 9:34 AM, Jakub Narebski wrote: Derrick Stolee writes: In the commit-graph file, the OID fanout chunk provides an index into the OID lookup. The 'verify' subcommand should find incorrect values in the fanout. Similarly, the 'verify' subcommand should find out-of-order values

Re: [PATCH 27/35] commit-slabs: remove realloc counter outside of slab struct

On 5/29/2018 8:48 PM, Stefan Beller wrote: The realloc counter is declared outside the struct for the given slabname, which makes it harder for a follow up patch to move the declaration of the struct around as then the counter variable would need special treatment. As the reallocation counter

Re: [RFC PATCH 00/35] object-store: lookup_commit

On 5/29/2018 11:18 PM, Stefan Beller wrote: On Tue, May 29, 2018 at 6:05 PM, Derrick Stolee wrote: On 5/29/2018 8:47 PM, Stefan Beller wrote: This applies on the merge of nd/commit-util-to-slab and sb/object-store-grafts, and is available at http://github.com/stefanbeller/ as branch object

Re: [RFC PATCH 00/35] object-store: lookup_commit

On 5/29/2018 8:47 PM, Stefan Beller wrote: This applies on the merge of nd/commit-util-to-slab and sb/object-store-grafts, and is available at http://github.com/stefanbeller/ as branch object-store-lookup-commit as the merge has some merge conflicts as well as syntactical conflicts

Re: [PATCH] commit-graph: fix a sparse 'integer as NULL pointer' warning

n 5/29/2018 4:01 PM, Ramsay Jones wrote: Signed-off-by: Ramsay Jones --- Hi Derrick, If you need to re-roll your 'ds/commit-graph-fsck' branch (pu@a84e06bc0f), could you please squash this into the relevant patch (commit 80453b4529, "commit-graph: add 'verify' subcommand", 2018-05-24). [No,

Re: [PATCH v3 07/20] commit-graph: verify catches corrupt signature

On 5/28/2018 10:05 AM, Jakub Narebski wrote: Derrick Stolee writes: This is the first of several commits that add a test to check that 'git commit-graph verify' catches corruption in the commit-graph file. The first test checks that the command catches an error in the file signature

Re: [PATCH v3 03/20] commit-graph: parse commit from chosen graph

On 5/27/2018 6:23 AM, Jakub Narebski wrote: Derrick Stolee writes: Before verifying a commit-graph file against the object database, we need to parse all commits from the given commit-graph file. Create parse_commit_in_graph_one() to target a given struct commit_graph. If I understand

Re: [PATCH] t990X: use '.git/objects' as 'deep inside .git' path

On 5/27/2018 12:49 AM, Michael Haggerty wrote: On Sat, May 26, 2018 at 8:47 AM, Christian Couder wrote: Tests t9902-completion.sh and t9903-bash-prompt.sh each have tests that check what happens when we are "in the '.git' directory" and when we are "deep inside the '.git' directory". To test

Re: [PATCH v3 00/20] Integrate commit-graph into 'fsck' and 'gc'

On 5/29/2018 12:27 AM, Junio C Hamano wrote: Derrick Stolee writes: Thanks for all the feedback on v2. I've tried to make this round's review a bit easier by splitting up the commits into smaller pieces. Also, the test script now has less boilerplate and uses variables and clear arithmetic

[RFC PATCH 5/6] fetch: destroy commit graph on shallow parameters

The commit-graph feature is not compatible with history-rewriting features such as shallow clones. When running a 'git fetch' with any of the shallow/unshallow options, destroy the commit-graph file and override core.commitGraph to be false. Signed-off-by: Derrick Stolee --- builtin/fetch.c | 6

[RFC PATCH 2/6] DO NOT MERGE: write commit-graph on every fetch

THIS IS ONLY FOR TESTING TO INCREASE TEST COVERAGE Signed-off-by: Derrick Stolee --- builtin/fetch.c | 4 1 file changed, 4 insertions(+) diff --git a/builtin/fetch.c b/builtin/fetch.c index 8ee998ea2e..af896e4b74 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -20,6 +20,7

[RFC PATCH 1/6] DO NOT MERGE: compute commit-graph on every commit

Also enable core.commitGraph and gc.commitGraph. Helps to test the commit-graph feature with the rest of the test infrastructure. Signed-off-by: Derrick Stolee --- builtin/commit.c | 5 + builtin/gc.c | 2 +- environment.c| 2 +- 3 files changed, 7 insertions(+), 2 deletions

[RFC PATCH 4/6] commit-graph: avoid writing when repo is shallow

Shallow clones do not interact well with the commit-graph feature for several reasons. Instead of doing the hard thing to fix those interactions, instead prevent reading or writing a commit-graph file for shallow repositories. Signed-off-by: Derrick Stolee --- commit-graph.c | 12

[RFC PATCH 6/6] commit-graph: revert to odb on missing parents

the commit-graph failed to fill the contents. Then the caller is responsible for filling the rest of the data from a commit buffer. Signed-off-by: Derrick Stolee --- commit-graph.c | 33 ++--- 1 file changed, 30 insertions(+), 3 deletions(-) diff --git a/commit-graph.

[RFC PATCH 3/6] commit-graph: enable replace-object and grafts

, then ignore the commit-graph for that commit and insted use the parents loaded by parsing the commit buffer and comparing against the graft file. Signed-off-by: Derrick Stolee --- builtin/replace.c | 3 +++ commit-graph.c| 20 +++- commit-graph.h| 9 + commit.c

[RFC PATCH 0/6] Fix commit-graph/graft/replace/shallow combo

while, I figured I should just put this series up for discussion. Maybe someone with more experience in shallow clones can point out the obvious issues I'm having. Thanks, -Stolee Derrick Stolee (6): DO NOT MERGE: compute commit-graph on every commit DO NOT MERGE: write commit-graph on ever

Re: [PATCH v3 16/20] commit-graph: verify contents match checksum

On 6/2/2018 11:52 AM, Jakub Narebski wrote: Derrick Stolee writes: The commit-graph file ends with a SHA1 hash of the previous contents. If a commit-graph file has errors but the checksum hash is correct, then we know that the problem is a bug in Git and not simply file corruption after

[PATCH] t5318-commit-graph.sh: use core.commitGraph

-by: Derrick Stolee --- t/t5318-commit-graph.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index 77d85aefe7..59d0be2877 100755 --- a/t/t5318-commit-graph.sh +++ b/t/t5318-commit-graph.sh @@ -28,8 +28,8 @@ test_expect_success

Re: [PATCH v3 09/20] commit-graph: verify corrupt OID fanout and lookup

On 6/2/2018 12:38 AM, Duy Nguyen wrote: On Thu, May 24, 2018 at 6:25 PM, Derrick Stolee wrote: + if (i && oidcmp(_oid, _oid) >= 0) + graph_report("commit-graph has incorrect OID order: %s then %s", +

Re: [PATCH v3 17/20] fsck: verify commit-graph

On 6/2/2018 12:17 PM, Jakub Narebski wrote: Derrick Stolee writes: If core.commitGraph is true, verify the contents of the commit-graph during 'git fsck' using the 'git commit-graph verify' subcommand. Run this check on all alternates, as well. All right, so we have one config variable

Re: [PATCH v3 15/20] commit-graph: test for corrupted octopus edge

On 6/2/2018 8:39 AM, Jakub Narebski wrote: Derrick Stolee writes: The commit-graph file has an extra chunk to store the parent int-ids for parents beyond the first parent for octopus merges. Our test repo has a single octopus merge that we can manipulate to demonstrate the 'verify' subcommand

Re: [PATCH v3 06/20] commit-graph: add 'verify' subcommand

On 6/2/2018 5:19 PM, Jakub Narebski wrote: Derrick Stolee writes: Do we have a way to run individual steps of the test suite? I am unfamiliar with that process. The t/README describes three such ways in "Skipping Tests" section: - GIT_SKIP_TESTS environment variable, which can

Re: [PATCH v3 19/20] gc: automatically write commit-graph files

On 6/2/2018 2:03 PM, Jakub Narebski wrote: Derrick Stolee writes: The commit-graph file is a very helpful feature for speeding up git operations. In order to make it more useful, write the commit-graph file by default during standard garbage collection operations. I think you meant here

Re: [PATCH v3 13/20] commit-graph: verify generation number

On 6/2/2018 8:23 AM, Jakub Narebski wrote: Derrick Stolee writes: While iterating through the commit parents, perform the generation number calculation and compare against the value stored in the commit-graph. All right, that's good. What about commit-graph files that have

Re: [RFC PATCH 0/6] Fix commit-graph/graft/replace/shallow combo

On 5/31/2018 2:33 PM, Stefan Beller wrote: On Thu, May 31, 2018 at 10:40 AM, Derrick Stolee wrote: The commit-graph file stores a condensed version of the commit history. This helps speed up several operations involving commit walks. This feature does not work well if those commits "c

Re: ds/generation-numbers (was Re: What's cooking in git.git (Jun 2018, #01; Fri, 1))

On 6/1/2018 7:15 PM, Junio C Hamano wrote: Derrick Stolee writes: A recently added "commit-graph" datafile has learned to store pre-computed generation numbers to speed up the decisions to stop history traversal. Will cook in 'next'. On Wednesday, these were marked as &

ds/generation-numbers (was Re: What's cooking in git.git (Jun 2018, #01; Fri, 1))

On 6/1/2018 3:21 AM, Junio C Hamano wrote: * ds/commit-graph-lockfile-fix (2018-05-22) 1 commit (merged to 'next' on 2018-05-24 at 3d12a02b0c) + commit-graph: fix UX issue when .lock file exists (this branch is used by ds/commit-graph-fsck; uses ds/generation-numbers.) Update to

Re: [RFC PATCH 4/6] commit-graph: avoid writing when repo is shallow

On 5/31/2018 10:30 PM, Junio C Hamano wrote: Derrick Stolee writes: Shallow clones do not interact well with the commit-graph feature for several reasons. Instead of doing the hard thing to fix those interactions, instead prevent reading or writing a commit-graph file for shallow repositories

Re: [PATCH v4 00/21] Integrate commit-graph into 'fsck' and 'gc'

On 6/5/2018 10:51 AM, Ævar Arnfjörð Bjarmason wrote: On Mon, Jun 4, 2018 at 6:52 PM, Derrick Stolee wrote: Thanks for the feedback on v3. There were several small cleanups, but perhaps the biggest change is the addition of "commit-graph: use string-list API for input" which makes &qu

Re: t5318-commit-graph.sh breaks travis gettext poison job

On 6/1/2018 12:17 PM, Duy Nguyen wrote: In case you're not checking travis, [1] reports Test Summary Report --- t5318-commit-graph.sh(Wstat: 256 Tests: 62 Failed: 2) Failed tests: 61-62 Non-zero exit status: 1 This usually means you're

Re: [PATCH v3 10/20] commit-graph: verify objects exist

On 5/30/2018 3:22 PM, Jakub Narebski wrote: Derrick Stolee writes: In the 'verify' subcommand, load commits directly from the object database to ensure they exist. Parse by skipping the commit-graph. All right, before we check that the commit data matches, we need to check that all

Re: [PATCH v3 11/20] commit-graph: verify root tree OIDs

On 5/30/2018 6:24 PM, Jakub Narebski wrote: Derrick Stolee writes: The 'verify' subcommand must compare the commit content parsed from the commit-graph and compare it against the content in the object database. You have "compare" twice in the above sentence. Use loo

[PATCH v4 06/21] commit-graph: add 'verify' subcommand

state. Do not report any errors. Helped-by: Ramsay Jones Signed-off-by: Derrick Stolee --- Documentation/git-commit-graph.txt | 6 + builtin/commit-graph.c | 38 ++ commit-graph.c | 23 ++ commit-graph.h

[PATCH v4 07/21] commit-graph: verify catches corrupt signature

that is intended for a typical user. Helped-by: Szeder Gábor Signed-off-by: Derrick Stolee --- t/t5318-commit-graph.sh | 43 + 1 file changed, 43 insertions(+) diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index 0830ef9fdd..c0c1ff09b9

[PATCH v4 03/21] commit-graph: parse commit from chosen graph

Before verifying a commit-graph file against the object database, we need to parse all commits from the given commit-graph file. Create parse_commit_in_graph_one() to target a given struct commit_graph. Signed-off-by: Derrick Stolee --- commit-graph.c | 18 +++--- 1 file changed, 15

[PATCH v4 08/21] commit-graph: verify required chunks are present

The commit-graph file requires the following three chunks: * OID Fanout * OID Lookup * Commit Data If any of these are missing, then the 'verify' subcommand should report a failure. This includes the chunk IDs malformed or the chunk count is truncated. Signed-off-by: Derrick Stolee --- commit

[PATCH v4 05/21] commit-graph: load a root tree from specific graph

When lazy-loading a tree for a commit, it will be important to select the tree from a specific struct commit_graph. Create a new method that specifies the commit-graph file and use that in get_commit_tree_in_graph(). Signed-off-by: Derrick Stolee --- commit-graph.c | 12 +--- 1 file

[PATCH v4 00/21] Integrate commit-graph into 'fsck' and 'gc'

large, but I'll send it in a follow-up PR. Thanks, -Stolee Derrick Stolee (21): commit-graph: UNLEAK before die() commit-graph: fix GRAPH_MIN_SIZE commit-graph: parse commit from chosen graph commit: force commit to parse from object database commit-graph: load a root tree from specific graph commi

[PATCH v4 11/21] commit-graph: verify root tree OIDs

the object database. Add checks for the root tree OID. Signed-off-by: Derrick Stolee --- commit-graph.c | 17 - t/t5318-commit-graph.sh | 7 +++ 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index 0cf1b61d80

[PATCH v4 09/21] commit-graph: verify corrupt OID fanout and lookup

In the commit-graph file, the OID fanout chunk provides an index into the OID lookup. The 'verify' subcommand should find incorrect values in the fanout. Similarly, the 'verify' subcommand should find out-of-order values in the OID lookup. Signed-off-by: Derrick Stolee --- commit-graph.c

[PATCH v4 12/21] commit-graph: verify parent list

from the object database. Test these checks for corrupt parents, too many parents, and wrong parents. Add a boundary check to insert_parent_or_die() for when the parent position value is out of range. The octopus merge will be tested in a later commit. Signed-off-by: Derrick Stolee --- commit

[PATCH v4 02/21] commit-graph: fix GRAPH_MIN_SIZE

the magic constants. Signed-off-by: Derrick Stolee --- commit-graph.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index bb54c1214c..c09e87c3c2 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -34,10 +34,11 @@ #define GRAPH_LAST_EDGE

[PATCH v4 01/21] commit-graph: UNLEAK before die()

Signed-off-by: Derrick Stolee --- builtin/commit-graph.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index 37420ae0fd..f0875b8bf3 100644 --- a/builtin/commit-graph.c +++ b/builtin/commit-graph.c @@ -51,8 +51,11 @@ static

[PATCH v4 10/21] commit-graph: verify objects exist

In the 'verify' subcommand, load commits directly from the object database to ensure they exist. Parse by skipping the commit-graph. Signed-off-by: Derrick Stolee --- commit-graph.c | 17 + t/t5318-commit-graph.sh | 7 +++ 2 files changed, 24 insertions(+) diff

[PATCH v4 15/21] commit-graph: test for corrupted octopus edge

The commit-graph file has an extra chunk to store the parent int-ids for parents beyond the first parent for octopus merges. Our test repo has a single octopus merge that we can manipulate to demonstrate the 'verify' subcommand detects incorrect values in that chunk. Signed-off-by: Derrick Stolee

[PATCH v4 04/21] commit: force commit to parse from object database

is explicit in avoiding commits from the commit-graph file. Signed-off-by: Derrick Stolee --- commit.c | 9 +++-- commit.h | 1 + 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/commit.c b/commit.c index 1d28677dfb..6eaed0174c 100644 --- a/commit.c +++ b/commit.c @@ -392,7

[PATCH v4 18/21] commit-graph: use string-list API for input

Signed-off-by: Derrick Stolee --- builtin/commit-graph.c | 39 +-- commit-graph.c | 15 +++ commit-graph.h | 7 +++ 3 files changed, 23 insertions(+), 38 deletions(-) diff --git a/builtin/commit-graph.c b/builtin/commit

[PATCH v4 13/21] commit-graph: verify generation number

have a mix of zero and non-zero generation numbers. Create a test that sets one commit to generation zero and all following commits report a failure as they have non-zero generation in a file that contains generation number zero. Signed-off-by: Derrick Stolee --- commit-graph.c | 34

<    1   2   3   4   5   6   7   8   9   10   >