[PATCH 14/17] commit-graph: load split commit-graph files

2019-05-08 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Starting with commit-graph, load commit-graph files in a sequence as follows: commit-graph commit-graph-1 commit-graph-2 ... commit-graph-N This creates N + 1 files in order. Signed-off-by: Derrick Stolee --- commit-graph.c | 39

[PATCH 01/17] commit-graph: fix the_repository reference

2019-05-08 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The parse_commit_buffer() method takes a repository pointer, so it should not refer to the_repository anymore. Signed-off-by: Derrick Stolee --- commit.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/commit.c b/commit.c index a5333c7ac6..e4d1233226

Re: [PATCH 01/19] revision.h: avoid bit fields in struct rev_info

2019-05-08 Thread Derrick Stolee
On 5/8/2019 10:41 AM, Duy Nguyen wrote: > On Wed, May 8, 2019 at 9:07 PM Derrick Stolee wrote: >> >> On 5/8/2019 7:12 AM, Nguyễn Thái Ngọc Duy wrote: >>> Bitfield addresses cannot be passed around in a pointer. This makes it >>> hard to use parse-options to se

Re: [PATCH 1/2] midx: pass a repository pointer

2019-05-08 Thread Derrick Stolee
On 5/7/2019 4:31 AM, Junio C Hamano wrote: > "Derrick Stolee via GitGitGadget" writes: > >> From: Derrick Stolee >> >> Much of the multi-pack-index code focuses on the multi_pack_index >> struct, and so we only pass a pointer to the current one. Howeve

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-08 Thread Derrick Stolee
On 5/8/2019 3:11 AM, Jeff King wrote: > On Tue, May 07, 2019 at 10:12:06AM +0200, Ævar Arnfjörð Bjarmason wrote: > >>> I think we'd want a way to tell the bitmap code to update our progress >>> meter as it traverses (both single objects, but also taking into account >>> when it finds a bitmap and

Re: [PATCH 01/19] revision.h: avoid bit fields in struct rev_info

2019-05-08 Thread Derrick Stolee
On 5/8/2019 7:12 AM, Nguyễn Thái Ngọc Duy wrote: > Bitfield addresses cannot be passed around in a pointer. This makes it > hard to use parse-options to set/unset them. Turn this struct to > normal integers. This of course increases the size of this struct > multiple times, but since we only have a

Re: [PATCH] commit-graph: fix memory leak

2019-05-06 Thread Derrick Stolee
On 5/6/2019 5:58 PM, Emily Shaffer wrote: > Hi, > > This change looks good to me, and like good evidence for the benefits of > automated tooling :) Same here! Keep up the great work here. -Stolee

Re: [PATCH v3 0/6] Create commit-graph file format v2

2019-05-06 Thread Derrick Stolee
On 5/6/2019 4:27 AM, Christian Couder wrote: > On Fri, May 3, 2019 at 3:44 PM Ævar Arnfjörð Bjarmason > wrote: >> 1) We can stat() the "commit-graphs" directory to see if there's any >>new/deleted ones (dir mtime changed), similar to what we do for the >>untracked cache, and can (but I do

Re: [PATCH v3 0/6] Create commit-graph file format v2

2019-05-03 Thread Derrick Stolee
On 5/3/2019 10:16 AM, SZEDER Gábor wrote: > On Fri, May 03, 2019 at 08:47:25AM -0400, Derrick Stolee wrote: >> It would be much simpler to restrict the model. Your idea of changing >> the file name is the inspiration here. >> >> * The "commit-graph" file i

Re: [PATCH v3 0/6] Create commit-graph file format v2

2019-05-03 Thread Derrick Stolee
On 5/2/2019 2:02 PM, Ævar Arnfjörð Bjarmason wrote: > > But those are separate from any back-compat concerns, which is what I > think makes sense to focus on now. Thinking more on this topic, I think I have a way to satisfy _all_ of your concerns by simplifying the plan for incremental commit-grap

Re: [PATCH v3 0/6] Create commit-graph file format v2

2019-05-02 Thread Derrick Stolee
On 5/1/2019 4:25 PM, Ævar Arnfjörð Bjarmason wrote: > I won't repeat my outstanding v2 feedback about v1 & v2 > incompatibilities, except to say that I'd in principle be fine with > having a v2 format the way this series is adding it. I.e. saying "here > it is, it's never written by default, we'll

Re: [PATCH v3 6/6] commit-graph: remove Future Work section

2019-05-01 Thread Derrick Stolee
On 5/1/2019 10:58 AM, Ævar Arnfjörð Bjarmason wrote: > > On Wed, May 01 2019, Derrick Stolee via GitGitGadget wrote: > >> The commit-graph feature began with a long list of planned >> benefits, most of which are now complete. The future work >> section has only a f

Re: [PATCH v3 5/6] commit-graph: implement file format version 2

2019-05-01 Thread Derrick Stolee
On 5/1/2019 3:12 PM, Ævar Arnfjörð Bjarmason wrote: > > OK, let's try that then, on top of this series: > > diff --git a/commit-graph.c b/commit-graph.c > index 5eebba6a0f..36c8cdb950 100644 > --- a/commit-graph.c > +++ b/commit-graph.c > @@ -1127,7 +1127,7 @@ int write_commit

[PATCH v3 6/6] commit-graph: remove Future Work section

2019-05-01 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The commit-graph feature began with a long list of planned benefits, most of which are now complete. The future work section has only a few items left. As for making more algorithms aware of generation numbers, some are only waiting for generation number v2 to ensure the

[PATCH v3 2/6] commit-graph: collapse parameters into flags

2019-05-01 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() and write_commit_graph_reachable() methods currently take two boolean parameters: 'append' and 'report_progress'. We will soon expand the possible options to send to these methods, so instead of complicating the parameter lis

[PATCH v3 5/6] commit-graph: implement file format version 2

2019-05-01 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The commit-graph file format had some shortcomings which we now correct: 1. The hash algorithm was determined by a single byte, instead of the 4-byte format identifier. 2. There was no way to update the reachability index we used. We currently only support

[PATCH v3 0/6] Create commit-graph file format v2

2019-05-01 Thread Derrick Stolee via GitGitGadget
unsigned int flags); int verify_commit_graph(struct repository *r, struct commit_graph *g); 3: 4ddb829163 ! 3: 8446011a43 commit-graph: create new version flags @@ -1,12 +1,12 @@ Author: Derrick Stolee -commit-graph: create new vers

[PATCH v3 4/6] commit-graph: add --version= option

2019-05-01 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Allow the commit-graph builtin to specify the file format version using the '--version=' option. Specify the version exactly in the verification tests as using a different version would change the offsets used in those tests. Signed-off-by: Derr

[PATCH v3 3/6] commit-graph: create new version parameter

2019-05-01 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee In anticipation of a new commit-graph file format version, create a parameter for the write_commit_graph() and write_commit_graph_reachable() methods to take a version number. When the given version is zero, the implementation selects a default value. Currently, the only

[PATCH v3 1/6] commit-graph: return with errors during write

2019-05-01 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method uses die() to report failure and exit when confronted with an unexpected condition. This use of die() in a library function is incorrect and is now replaced by error() statements and an int return type. Now that we use 'goto cleanup

Re: [PATCH 3/6] config.c: add repo_config_set_worktree_gently()

2019-04-30 Thread Derrick Stolee
On 12/27/2018 10:56 AM, Nguyễn Thái Ngọc Duy wrote: > diff --git a/config.h b/config.h > index ee5d3fa7b4..62204dc252 100644 > --- a/config.h > +++ b/config.h > @@ -103,6 +103,9 @@ extern int git_config_color(char *, const char *, const > char *); > extern int git_config_set_in_file_gently(const

[PATCH 0/2] Multi-pack-index: Fix "too many file descriptors" bug

2019-04-29 Thread Derrick Stolee via GitGitGadget
flicts if I pick only Jeff's change as a base: jh/midx-verify-too-many-packs jk/server-info-rabbit-hole bc/hash-transition-16 Thanks, -Stolee Derrick Stolee (2): midx: pass a repository pointer midx: add packs to packed_git linked list builtin/multi-pack-index.c | 2 +- builtin/pack-obje

[PATCH 1/2] midx: pass a repository pointer

2019-04-29 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Much of the multi-pack-index code focuses on the multi_pack_index struct, and so we only pass a pointer to the current one. However, we will insert a dependency on the packed_git linked list in a future change, so we will need a repository reference. Inserting these

[PATCH 2/2] midx: add packs to packed_git linked list

2019-04-29 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The multi-pack-index allows searching for objects across multiple packs using one object list. The original design gains many of these performance benefits by keeping the packs in the multi-pack-index out of the packed_git list. Unfortunately, this has one major drawback

Re: [PATCH v3 4/8] commit-graph: don't early exit(1) on e.g. "git status"

2019-04-29 Thread Derrick Stolee
On 4/27/2019 9:06 AM, Ævar Arnfjörð Bjarmason wrote: > > There's still cases left where we'll exit early, e.g. if you do: > > $ git diff -U1 > diff --git a/commit-graph.c b/commit-graph.c > index 66865acbd7..63773764ce 100644 > --- a/commit-graph.c > +++ b/commit-graph.c >

Re: [PATCH v2 0/5] Create commit-graph file format v2

2019-04-26 Thread Derrick Stolee
On 4/26/2019 4:33 AM, Ævar Arnfjörð Bjarmason wrote: > > On Fri, Apr 26 2019, Junio C Hamano wrote: >> >> Thanks always for your careful review and thoughtful comments, by >> the way. I agree that these comments are extremely helpful. >>> Now as noted in my series we now on 'master' downgrade th

[PATCH 0/1] commit-graph: improve error messages

2019-04-26 Thread Derrick Stolee via GitGitGadget
-Reply-To: 878svxrwsh@evledraar.gmail.com [878svxrwsh@evledraar.gmail.com] Derrick Stolee (1): commit-graph: improve error messages commit-graph.c | 10 +- t/t5318-commit-graph.sh | 6 +++--- 2 files changed, 8 insertions(+), 8 deletions(-) base-commit

[PATCH 1/1] commit-graph: improve error messages

2019-04-26 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The error messages when reading a commit-graph have a few problems: 1. Some values are output in hexadecimal, but that is not made clear by the message. Prepend "0x" to these values. 2. The version number does not need to be hexadecimal, and also should

Re: [PATCH] trace2: fix incorrect function pointer check

2019-04-25 Thread Derrick Stolee
On 4/25/2019 1:08 PM, Josh Steadmon wrote: > Fix trace2_data_json_fl() to check for the presence of pfn_data_json_fl > in its targets, rather than pfn_data_fl, which is not actually called. [snip] > for_each_wanted_builtin (j, tgt_j) > - if (tgt_j->pfn_data_fl) > + i

Git Test Coverage Report (April 25)

2019-04-25 Thread Derrick Stolee
romisor_remote() Christian Couder dcc8b4e9 promisor-remote: implement promisor_remote_get_direct() Christian Couderb21a55f3 promisor-remote: parse remote.*.partialclonefilter Christian Couder7b6e1b04 Move core_partial_clone_filter_default to promisor-remote.c Christian Couder

Re: [PATCH v2 3/5] commit-graph: create new version flags

2019-04-25 Thread Derrick Stolee
On 4/25/2019 1:29 AM, Junio C Hamano wrote: > "Derrick Stolee via GitGitGadget" writes: > >> +int version = 0; >> ... >> +if (flags & COMMIT_GRAPH_VERSION_1) >> +version = 1; >> +if (!version) >> +versio

Re: [PATCH v5 00/11] Create 'expire' and 'repack' verbs for git-multi-pack-index

2019-04-25 Thread Derrick Stolee
On 4/25/2019 1:38 AM, Junio C Hamano wrote: > Derrick Stolee writes: > >> Updates in V5: >> >> * Fixed the error in PATCH 7 due to a missing line that existed in PATCH 8. >> Thanks, Josh Steadmon! >> >> * The 'repack' subcommand now c

[PATCH v2 4/5] commit-graph: add --version= option

2019-04-24 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Allow the commit-graph builtin to specify the file format version using the '--version=' option. Specify the version exactly in the verification tests as using a different version would change the offsets used in those tests. Signed-off-by: Derr

[PATCH v2 2/5] commit-graph: collapse parameters into flags

2019-04-24 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() and write_commit_graph_reachable() methods currently take two boolean parameters: 'append' and 'report_progress'. We will soon expand the possible options to send to these methods, so instead of complicating the parameter lis

[PATCH v2 1/5] commit-graph: return with errors during write

2019-04-24 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method uses die() to report failure and exit when confronted with an unexpected condition. This use of die() in a library function is incorrect and is now replaced by error() statements and an int return type. Now that we use 'goto cleanup

[PATCH v2 5/5] commit-graph: implement file format version 2

2019-04-24 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The commit-graph file format had some shortcomings which we now correct: 1. The hash algorithm was determined by a single byte, instead of the 4-byte format identifier. 2. There was no way to update the reachability index we used. We currently only support

[PATCH v2 3/5] commit-graph: create new version flags

2019-04-24 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee In anticipation of a new commit-graph file format version, create a flag for the write_commit_graph() and write_commit_graph_reachable() methods to take a version number. When there is no specified version, the implementation selects a default value. Currently, the only

[PATCH v2 0/5] Create commit-graph file format v2

2019-04-24 Thread Derrick Stolee via GitGitGadget
ublic-inbox.org/git/6367e30a-1b3a-4fe9-611b-d931f51ef...@gmail.com/ Derrick Stolee (5): commit-graph: return with errors during write commit-graph: collapse parameters into flags commit-graph: create new version flags commit-graph: add --version= option commit-graph: implement file format vers

[PATCH v5 08/11] midx: implement midx_repack()

2019-04-24 Thread Derrick Stolee
l size of the referenced objects, or can create a large pack if the actual size of the referenced objects is larger than the expected size. Signed-off-by: Derrick Stolee --- midx.c | 150 +++- t/t5319-multi-pack-index.sh | 28 +++ 2 file

[PATCH v5 01/11] repack: refactor pack deletion for future use

2019-04-24 Thread Derrick Stolee
enabled for the use in repack, but will be important for a future caller. Signed-off-by: Derrick Stolee --- builtin/repack.c | 14 ++ packfile.c | 28 packfile.h | 7 +++ 3 files changed, 37 insertions(+), 12 deletions(-) diff --git

[PATCH v5 02/11] Docs: rearrange subcommands for multi-pack-index

2019-04-24 Thread Derrick Stolee
Gábor Signed-off-by: Derrick Stolee --- Documentation/git-multi-pack-index.txt | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/Documentation/git-multi-pack-index.txt b/Documentation/git-multi-pack-index.txt index f7778a2c85..1af406aca2 100644 --- a/Documentation/git-

[PATCH v5 06/11] multi-pack-index: implement 'expire' subcommand

2019-04-24 Thread Derrick Stolee
ndex to ensure it no longer references those packs. Signed-off-by: Derrick Stolee --- midx.c | 120 +--- t/t5319-multi-pack-index.sh | 20 ++ 2 files changed, 130 insertions(+), 10 deletions(-) diff --git a/midx.c b/midx.c index 95c39106b2..299

[PATCH v5 07/11] multi-pack-index: prepare 'repack' subcommand

2019-04-24 Thread Derrick Stolee
rted by size (ascending). This will be important in a future test. Signed-off-by: Derrick Stolee --- Documentation/git-multi-pack-index.txt | 17 + builtin/multi-pack-index.c | 12 ++-- midx.c | 5 + midx.h

[PATCH v5 09/11] multi-pack-index: test expire while adding packs

2019-04-24 Thread Derrick Stolee
the multi-pack-index. Add a test to t5319-multi-pack-index.sh that catches this issue. The test adds new packs that cause another pack to be expired, and creates new packs that are lexicographically sorted before and after the existing packs. Signed-off-by: Derrick Stolee --- t/t5319-multi-pack

[PATCH v5 10/11] midx: add test that 'expire' respects .keep files

2019-04-24 Thread Derrick Stolee
pack that contains every object in the repo, then add it to the multi-pack-index. Then create a .keep file for a pack starting with "a-pack" that was added in the previous test. Finally, expire and verify that the pack remains and the other packs were expired. Signed-off-by: Derrick

[PATCH v5 00/11] Create 'expire' and 'repack' verbs for git-multi-pack-index

2019-04-24 Thread Derrick Stolee
a test that hits the boundary cases here, but follows the 'expire' subcommand so we can show that cycle of repack-then-expire to safely replace the packs. Junio: It appears that there are some conflicts with the trace2 changes in master. These are not new to the updates in this version. I

[PATCH v5 11/11] t5319-multi-pack-index.sh: test batch size zero

2019-04-24 Thread Derrick Stolee
pack-files, as they no longer contain any referenced objects in the multi-pack-index. We must remove the .keep file that was added in the previous test in order to expire that pack-file. Also test that a 'repack' will do nothing if there is only one pack-file. Signed-off-by: Derrick Stole

[PATCH v5 05/11] midx: refactor permutation logic and pack sorting

2019-04-24 Thread Derrick Stolee
ere tracking in that struct. Finally, update get_sorted_entries() to not permute the pack-int-id and instead supply the permutation to write_midx_object_offsets(). This requires sorting the packs after get_sorted_entries(). Signed-off-by: Derrick Stole

[PATCH v5 04/11] midx: simplify computation of pack name lengths

2019-04-24 Thread Derrick Stolee
g the file. Signed-off-by: Derrick Stolee --- midx.c | 18 +- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/midx.c b/midx.c index bb825ef816..f087bbbe82 100644 --- a/midx.c +++ b/midx.c @@ -383,7 +383,6 @@ struct pack_list { uint32_t nr; uint32_t

[PATCH v5 03/11] multi-pack-index: prepare for 'expire' subcommand

2019-04-24 Thread Derrick Stolee
hey have a specific order when sorted by size. This will be important in a later test. Signed-off-by: Derrick Stolee --- Documentation/git-multi-pack-index.txt | 5 +++ builtin/multi-pack-index.c | 4 ++- midx.c | 5 +++ midx.h

Git Test Coverage Report (April 23)

2019-04-23 Thread Derrick Stolee
o allocate %" PRIuMAX " bytes)", 5efde212 73) error("Out of memory, malloc failed (tried to allocate %" PRIuMAX " bytes)", Commits introducting uncovered code: Barret Rhoden dfb4ee12 blame: use a fingerprint heuristic to match ignored lines Barret Rhoden d0

Re: [PATCH] t5304: add a test for pruning with bitmaps

2019-04-20 Thread Derrick Stolee
On 4/19/2019 11:24 PM, Jeff King wrote: > Try running t5304 with this: > > diff --git a/reachable.c b/reachable.c > index eba6f64e01..7ec73ef43f 100644 > --- a/reachable.c > +++ b/reachable.c > @@ -191,6 +191,8 @@ static int mark_object_seen(const struct object_id *oid, > if (!obj) >

Re: [PATCH] t5304: add a test for pruning with bitmaps

2019-04-19 Thread Derrick Stolee
On 4/18/2019 4:08 PM, Jeff King wrote: > On Thu, Apr 18, 2019 at 03:49:53PM -0400, Jeff King wrote: >> I dunno. I guess it does not hurt to at least to at least make sure this >> code is running in the normal suite. I don't think that will find the >> more interesting regressions, but at least save

Stalled ds/ branches (was What's cooking in git.git (Apr 2019, #03; Tue, 16))

2019-04-16 Thread Derrick Stolee
On 4/16/2019 9:19 AM, Junio C Hamano wrote: > * ds/midx-expire-repack (2019-01-27) 10 commits > - midx: add test that 'expire' respects .keep files > - multi-pack-index: test expire while adding packs > - midx: implement midx_repack() > - multi-pack-index: prepare 'repack' subcommand > - mult

Re: [PATCH 07/14] parse-options: allow ll_callback with OPTION_CALLBACK

2019-04-16 Thread Derrick Stolee
On 4/16/2019 4:52 AM, Duy Nguyen wrote: > On Mon, Apr 15, 2019 at 9:06 PM Derrick Stolee wrote: >> >> On 1/26/2019 7:35 PM, Nguyễn Thái Ngọc Duy wrote: >>> @@ -238,7 +249,10 @@ static enum parse_opt_result parse_short_opt(struct >>> parse_opt_ctx_t

Re: [PATCH 2/3] prune: use bitmaps for reachability traversal

2019-04-15 Thread Derrick Stolee
On 2/13/2019 11:37 PM, Jeff King wrote: > +static void *lookup_object_by_type(struct repository *r, > +const struct object_id *oid, > +enum object_type type) > +{ > + switch (type) { > + case OBJ_COMMIT: > + return

Re: [PATCH v6 2/2] config: allow giving separate author and committer idents

2019-04-15 Thread Derrick Stolee
On 2/5/2019 2:52 PM, Ævar Arnfjörð Bjarmason wrote: > From: William Hubbs > -const char *fmt_name(const char *name, const char *email) > +const char *fmt_name(enum want_ident whose_ident) > { > - return fmt_ident(name, email, NULL, IDENT_STRICT | IDENT_NO_DATE); > + char *name = NULL; > +

Re: [PATCH 07/14] parse-options: allow ll_callback with OPTION_CALLBACK

2019-04-15 Thread Derrick Stolee
On 1/26/2019 7:35 PM, Nguyễn Thái Ngọc Duy wrote: > @@ -238,7 +249,10 @@ static enum parse_opt_result parse_short_opt(struct > parse_opt_ctx_t *p, > len++; > arg = xmemdupz(p->opt, len); > p->opt = p->opt[len] ? p->opt + len : NULL; > -

Re: straw poll: git merge conference location

2019-03-14 Thread Derrick Stolee
On 3/13/2019 4:55 PM, Jeff King wrote: > We're looking at doing it in North America, but there are two specific > questions: > > - is there preference between East Coast vs West Coast? I have no preference here. > - preferences between Canada and US? There should be serious consideration fo

Re: [PATCH 2/4] revision walk: optionally use sparse reachability

2019-03-12 Thread Derrick Stolee
On 3/12/2019 9:18 AM, Nathaniel Filardo wrote: > The only caller that passes a non-zero value to prepare_revision_walk > after this patch is builtin/pack-objects. Without this, sparsity seems > to do little good therein, as prepare_revision_walk will densely > propagate UNINTERESTING flags from tr

Re: [PATCH 3/4] repack: add --sparse and pass to pack-objects

2019-03-12 Thread Derrick Stolee
On 3/12/2019 9:18 AM, Nathaniel Filardo wrote: > The sparse connectivity algorithm saves a whole lot of time when there > are UNINTERESTING trees around. Interesting! Do you have some performance numbers to include with this statement? > @@ -48,6 +49,10 @@ static int repack_config(const char *var,

[PATCH v7 10/15] trace2:data: pack-objects: add trace2 regions

2019-02-22 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When studying the performance of 'git push' we would like to know how much time is spent at various parts of the command. One area that could cause performance trouble is 'git pack-objects'. Add trace2 regions around the three main actions taken

Re: How to get next commit that just after a specified commit

2019-02-18 Thread Derrick Stolee
On 2/13/2019 4:36 PM, Ævar Arnfjörð Bjarmason wrote: > > On Wed, Feb 13 2019, Junio C Hamano wrote: > >> Ævar Arnfjörð Bjarmason writes: > > Their point in comparing it to git is that there's nothing intuitive in > the UI that exposes this information for the *current* graph, whereas in > fossi

[PATCH v6 10/15] trace2:data: pack-objects: add trace2 regions

2019-02-06 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When studying the performance of 'git push' we would like to know how much time is spent at various parts of the command. One area that could cause performance trouble is 'git pack-objects'. Add trace2 regions around the three main actions taken

Git Test Coverage Report (Wednesday, Feb. 6)

2019-02-06 Thread Derrick Stolee
_name, repo, &us_elapsed_absolute, 3c543ab3 458) strbuf_release(&buf_payload); 3c543ab3 459) } 3c543ab3 461) static void fn_data_fl(const char *file, int line, uint64_t us_elapsed_absolute, 3c543ab3 466) const char *event_name = "data"; 3c543ab3 467) struct strbuf buf_payload =

[PATCH v5 10/15] trace2:data: pack-objects: add trace2 regions

2019-02-01 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When studying the performance of 'git push' we would like to know how much time is spent at various parts of the command. One area that could cause performance trouble is 'git pack-objects'. Add trace2 regions around the three main actions taken

[PATCH v4 10/14] pack-objects: add trace2 regions

2019-01-30 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When studying the performance of 'git push' we would like to know how much time is spent at various parts of the command. One area that could cause performance trouble is 'git pack-objects'. Add trace2 regions around the three main actions taken

[PATCH v3 10/14] pack-objects: add trace2 regions

2019-01-30 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When studying the performance of 'git push' we would like to know how much time is spent at various parts of the command. One area that could cause performance trouble is 'git pack-objects'. Add trace2 regions around the three main actions taken

Re: [PATCH 1/1] Makefile: add prove and coverage-prove targets

2019-01-29 Thread Derrick Stolee
On 1/29/2019 1:10 PM, Derrick Stolee wrote: > On 1/29/2019 12:34 PM, SZEDER Gábor wrote: >> On Tue, Jan 29, 2019 at 04:58:27PM +0100, SZEDER Gábor wrote: >> And in the related email discussion [1]: >> >> But even though the docs claim it [-j] should be possible, >

Re: Contributor Summit Topics and Logistics

2019-01-29 Thread Derrick Stolee
I was hoping to attend the contributors' summit remotely, but now my leave is starting before then. This email contains a summary of what I would have added to the discussion. Thanks, -Stolee Commit-Graph Status Report == I'm really happy with the progress in this area,

Re: [PATCH 1/1] Makefile: add prove and coverage-prove targets

2019-01-29 Thread Derrick Stolee
On 1/29/2019 12:34 PM, SZEDER Gábor wrote: > On Tue, Jan 29, 2019 at 04:58:27PM +0100, SZEDER Gábor wrote: > And in the related email discussion [1]: > > But even though the docs claim it [-j] should be possible, > I've been getting "random" test failures when compiled with coverage > suppor

Git Test Coverage Report (Tue. January 29, 2019)

2019-01-29 Thread Derrick Stolee
Here is today's test coverage report. Since the last report, I have overhauled the machinery for generating these reports. The code is available on GitHub [1], and the reports are available online in text [2] and HTML [3] form. In addition to the two output formats, the code also has the ability

[PATCH v2 1/1] Makefile: add coverage-prove target

2019-01-29 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Sometimes there are test failures in the 'pu' branch. This is somewhat expected for a branch that takes the very latest topics under development, and those sometimes have semantic conflicts that only show up during test runs. This also can happen when running the

[PATCH v2 0/1] Makefile: add prove and coverage-prove targets

2019-01-29 Thread Derrick Stolee via GitGitGadget
even with a failure. If/when this merges down, I will use this new target for the automation around the test coverage reports. Updates in V2: * Dropped the 'prove' target * Append '-j1' to GIT_PROVE_OPTS * Commit message tweaks. Derrick Stolee (1): Makef

Re: [PATCH 1/1] Makefile: add prove and coverage-prove targets

2019-01-29 Thread Derrick Stolee
On 1/29/2019 10:58 AM, SZEDER Gábor wrote: > On Tue, Jan 29, 2019 at 06:56:08AM -0800, Derrick Stolee via GitGitGadget > wrote: >> +prove: all >> +$(MAKE) -C t/ prove >> + > > You don't need this 'prove' target in the "main" Makefi

Re: [PATCH 1/1] Makefile: add prove and coverage-prove targets

2019-01-29 Thread Derrick Stolee
On 1/29/2019 11:00 AM, Jeff King wrote: > On Tue, Jan 29, 2019 at 06:56:08AM -0800, Derrick Stolee via GitGitGadget > wrote: > >> From: Derrick Stolee >> >> When running the test suite for code coverage using >> 'make coverage-test', a single test fail

[PATCH 1/1] Makefile: add prove and coverage-prove targets

2019-01-29 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When running the test suite for code coverage using 'make coverage-test', a single test failure stops the test suite from completing. This leads to significant undercounting of covered blocks. Add two new targets to the Makefile: * 'prove' runs the tes

[PATCH 0/1] Makefile: add prove and coverage-prove targets

2019-01-29 Thread Derrick Stolee via GitGitGadget
even with a failure. If/when this merges down, I will use this new target for the automation around the test coverage reports. Thanks, -Stolee Derrick Stolee (1): Makefile: add prove and coverage-prove targets Makefile | 7 +++ 1 file changed, 7 insertions(+) base-commit: 0d0ac3826a3

[PATCH v2 10/14] pack-objects: add trace2 regions

2019-01-28 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When studying the performance of 'git push' we would like to know how much time is spent at various parts of the command. One area that could cause performance trouble is 'git pack-objects'. Add trace2 regions around the three main actions taken

Re: [PATCH] object_as_type: initialize commit-graph-related fields of 'struct commit'

2019-01-27 Thread Derrick Stolee
On 1/27/2019 8:28 AM, SZEDER Gábor wrote: > On Sun, Jan 27, 2019 at 02:08:32PM +0100, SZEDER Gábor wrote: >> When the commit graph and generation numbers were introduced in >> commits 177722b344 (commit: integrate commit graph with commit >> parsing, 2018-04-10) and 83073cc994 (commit: add generati

Re: [PATCH v4 08/10] midx: implement midx_repack()

2019-01-26 Thread Derrick Stolee
On 1/24/2019 4:52 PM, Derrick Stolee via GitGitGadget wrote: diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh index acc5e65ecc..d6c1353514 100755 --- a/t/t5319-multi-pack-index.sh +++ b/t/t5319-multi-pack-index.sh @@ -383,7 +383,8 @@ test_expect_success 'setup expire

Re: [PATCH v2] log,diff-tree: add --combined-with-paths options for merges with renames

2019-01-25 Thread Derrick Stolee
On 1/25/2019 11:54 AM, Elijah Newren wrote: > +test_expect_success '--combined-with-paths works with -z as well' ' > + printf "0f9645804ebb04cc3eef91f799eb7fb54d70cefb\0::100644 100644 > 100644 f00c965d8307308469e537302baa73048488f162 > 088bd5d92c2a8e0203ca8e7e4c2a5c692f6ae3f7 > 333b9c62519f

Re: [PATCH] log,diff-tree: add --combined-with-paths options for merges with renames

2019-01-25 Thread Derrick Stolee
On 1/24/2019 11:46 AM, Elijah Newren wrote: > As an alternative, I considered perhaps trying to sell it as a bugfix > (how often do people use -M, -c, and --raw together and have renames > in merge commits -- can I just change the format to include the old > names), but was worried that since diff-

Re: [PATCH 0/6] Create commit-graph file format v2

2019-01-25 Thread Derrick Stolee
On 1/24/2019 6:39 PM, Junio C Hamano wrote: > "Derrick Stolee via GitGitGadget" writes: > >> This series is based on ab/commit-graph-write-progress and bc/sha-256. > > Thanks. > > It seems that the base (i.e. merge between these two topics) you > used ma

[PATCH v4 08/10] midx: implement midx_repack()

2019-01-24 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee To repack using a multi-pack-index, first sort all pack-files by their modified time. Second, walk those pack-files from oldest to newest, adding the packs to a list if they are smaller than the given pack-size. Finally, collect the objects from the multi-pack- index that

[PATCH v4 10/10] midx: add test that 'expire' respects .keep files

2019-01-24 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The 'git multi-pack-index expire' subcommand may delete packs that are not needed from the perspective of the multi-pack-index. If a pack has a .keep file, then we should not delete that pack. Add a test that ensures we preserve a pack that would otherwise be expi

[PATCH v4 06/10] multi-pack-index: implement 'expire' subcommand

2019-01-24 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The 'git multi-pack-index expire' subcommand looks at the existing mult-pack-index, counts the number of objects referenced in each pack-file, deletes the pack-fils with no referenced objects, and rewrites the multi-pack-index to no longer reference those packs.

[PATCH v4 07/10] multi-pack-index: prepare 'repack' subcommand

2019-01-24 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee In an environment where the multi-pack-index is useful, it is due to many pack-files and an inability to repack the object store into a single pack-file. However, it is likely that many of these pack-files are rather small, and could be repacked into a slightly larger pack

[PATCH v4 02/10] Docs: rearrange subcommands for multi-pack-index

2019-01-24 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee We will add new subcommands to the multi-pack-index, and that will make the documentation a bit messier. Clean up the 'verb' descriptions by renaming the concept to 'subcommand' and removing the reference to the object directory. Helped-by: Stefan Belle

[PATCH v4 03/10] multi-pack-index: prepare for 'expire' subcommand

2019-01-24 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The multi-pack-index tracks objects in a collection of pack-files. Only one copy of each object is indexed, using the modified time of the pack-files to determine tie-breakers. It is possible to have a pack-file with no referenced objects because all objects have a duplicate

[PATCH v4 05/10] midx: refactor permutation logic and pack sorting

2019-01-24 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee In anticipation of the expire subcommand, refactor the way we sort the packfiles by name. This will greatly simplify our approach to dropping expired packs from the list. First, create 'struct pack_info' to replace 'struct pack_pair'. This struct co

[PATCH v4 04/10] midx: simplify computation of pack name lengths

2019-01-24 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Before writing the multi-pack-index, we compute the length of the pack-index names concatenated together. This forms the data in the pack name chunk, and we precompute it to compute chunk offsets. The value is also modified to fit alignment needs. Previously, this

[PATCH v4 09/10] multi-pack-index: test expire while adding packs

2019-01-24 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee During development of the multi-pack-index expire subcommand, a version went out that improperly computed the pack order if a new pack was introduced while other packs were being removed. Part of the subtlety of the bug involved the new pack being placed before other packs

[PATCH v4 01/10] repack: refactor pack deletion for future use

2019-01-24 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The repack builtin deletes redundant pack-files and their associated .idx, .promisor, .bitmap, and .keep files. We will want to re-use this logic in the future for other types of repack, so pull the logic into 'unlink_pack_path()' in packfile.c. The 'ignore_

[PATCH v4 00/10] Create 'expire' and 'repack' verbs for git-multi-pack-index

2019-01-24 Thread Derrick Stolee via GitGitGadget
quired rearranging the packs slightly to get different sizes than we had before. Also, I added 'touch -t' to set the modified times so we can fix the order in which the packs are selected. * Added a comment about the purpose of pack_perm. Thanks, -Stolee Derrick

Re: [PATCH v3 7/9] multi-pack-index: prepare 'repack' subcommand

2019-01-24 Thread Derrick Stolee
On 1/23/2019 5:38 PM, Jonathan Tan wrote: diff --git a/Documentation/git-multi-pack-index.txt b/Documentation/git-multi-pack-index.txt index 6186c4c936..cc63531cc0 100644 --- a/Documentation/git-multi-pack-index.txt +++ b/Documentation/git-multi-pack-index.txt @@ -36,6 +36,17 @@ expire::

Re: Git Test Coverage Report (Sat Jan 19)

2019-01-24 Thread Derrick Stolee
On 1/24/2019 1:15 PM, Junio C Hamano wrote: Derrick Stolee writes: Here is today's test coverage report. Also, there has been some feedback that it can be hard to manually match up uncovered lines with names at the bottom of the summary. The suggestion was to auto-generate an HTML r

Re: [PATCH v3 5/9] midx: refactor permutation logic and pack sorting

2019-01-24 Thread Derrick Stolee
On 1/24/2019 12:34 PM, Derrick Stolee wrote: On 1/23/2019 4:00 PM, Jonathan Tan wrote: Indeed, the sorting of pack_info is moved to after get_sorted_entries(). Also, pack_perm[old number] = new number, as expected. Thanks for chiming in with all the detail on the use of 'perm'. Th

Re: [PATCH v3 6/9] multi-pack-index: implement 'expire' verb

2019-01-24 Thread Derrick Stolee
On 1/23/2019 5:13 PM, Jonathan Tan wrote: Maybe add a fsck at the end for sanity's sake. Also, I think that preservation of .keep packfiles is an important feature, and maybe worth a test. Good points! I forgot to test the .keep stuff directly here because I have an equivalent test in VFS for Gi

Re: [PATCH v3 5/9] midx: refactor permutation logic and pack sorting

2019-01-24 Thread Derrick Stolee
On 1/23/2019 4:00 PM, Jonathan Tan wrote: Indeed, the sorting of pack_info is moved to after get_sorted_entries(). Also, pack_perm[old number] = new number, as expected. Thanks for chiming in with all the detail on the use of 'perm'. This is the most confusing part of this code path. I thin

<    2   3   4   5   6   7   8   9   10   11   >