Re: [PATCH v4 03/10] commit-graph: compute generation numbers

2018-04-26 Thread Derrick Stolee
On 4/26/2018 8:58 AM, Derrick Stolee wrote: n 4/25/2018 10:35 PM, Junio C Hamano wrote: Derrick Stolee <dsto...@microsoft.com> writes: @@ -439,6 +439,9 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len,   else   packedDate[

[PATCH] commit-graph: fix documentation inconsistencies

2018-06-28 Thread Derrick Stolee
in the documentation, but unfortunately reduces the number of commits that we can store in the commit-graph. Update that estimate, too. Reported-by: Grant Welch Signed-off-by: Derrick Stolee --- Documentation/technical/commit-graph-format.txt | 10 +- 1 file changed, 5 insertions(+),

Re: Inconsistencies in commit-graph technical docs.

2018-06-28 Thread Derrick Stolee
On 6/28/2018 1:11 AM, Grant Welch wrote: I recently read the "Supercharging the Git Commit Graph blog by Derrick Stolee. I found the article interesting and wanted to verify the performance numbers for myself. Then that led me to want to know more about the implementation, so I

Re: [PATCH v3 10/32] commit: add repository argument to parse_commit_buffer

2018-06-29 Thread Derrick Stolee
On 6/28/2018 9:22 PM, Stefan Beller wrote: Add a repository argument to allow the callers of parse_commit_buffer to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet. As

Re: [PATCH v3 09/32] commit: add repository argument to lookup_commit

2018-06-29 Thread Derrick Stolee
On 6/28/2018 9:21 PM, Stefan Beller wrote: Add a repository argument to allow callers of lookup_commit to be more specific about which repository to handle. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet. As with the

Re: [PATCH v3 00/32] object-store: lookup_commit

2018-06-29 Thread Derrick Stolee
On 6/28/2018 9:21 PM, Stefan Beller wrote: This continues the elimination of global variables in the object store and teaches lookup_commit[_reference] and alike to handle a_repository. This is also available as https://github.com/stefanbeller/git/tree/object-store-lookup-commit or applies on

[RFC PATCH 00/13] Consolidate reachability logic

2018-06-29 Thread Derrick Stolee
il the lookup_commit() and parse_commit() code is stable again. Thanks, -Stolee Derrick Stolee (13): commit-reach: move walk methods from commit.c commit-reach: move ref_newer from remote.c commit-reach: move commit_contains from ref-filter upload-pack: make reachable() more generic upload-pack: re

[RFC PATCH 06/13] commit-reach: move can_all_from_reach_with_flag()

2018-06-29 Thread Derrick Stolee
Signed-off-by: Derrick Stolee --- commit-reach.c | 61 + commit-reach.h | 8 +++ upload-pack.c | 62 +++--- 3 files changed, 72 insertions(+), 59 deletions(-) diff --git a/commit-reach.c b/commit

[RFC PATCH 01/13] commit-reach: move walk methods from commit.c

2018-06-29 Thread Derrick Stolee
Signed-off-by: Derrick Stolee --- Makefile | 1 + commit-reach.c | 359 + commit-reach.h | 41 ++ commit.c | 358 4 files changed, 401 insertions(+), 358 deletions(-) create mode

[RFC PATCH 02/13] commit-reach: move ref_newer from remote.c

2018-06-29 Thread Derrick Stolee
Signed-off-by: Derrick Stolee --- builtin/remote.c | 1 + commit-reach.c | 52 commit-reach.h | 2 ++ http-push.c | 1 + remote.c | 48 +--- remote.h | 1 - 6 files changed, 57

Re: [PATCH v7 07/22] commit-graph: add 'verify' subcommand

2018-06-29 Thread Derrick Stolee
On 6/27/2018 5:59 PM, Jonathan Tan wrote: +int verify_commit_graph(struct repository *r, struct commit_graph *g) I haven't had the time to review this patch set, but I did rebase my object store refactoring [1] on this and wrote a test: static void test_verify_commit_graph(const char

Re: What's cooking in git.git (Jun 2018, #07; Thu, 28)

2018-06-29 Thread Derrick Stolee
On 6/28/2018 6:42 PM, Stefan Beller wrote: On Thu, Jun 28, 2018 at 2:40 PM Junio C Hamano wrote: * ds/commit-graph-fsck (2018-06-27) 22 commits [...] "git fsck" learns to make sure the optional commit-graph file is in a sane state. Is this ready for 'next'? I hope so, as I plan to

Re: Git Test Coverage Report (Tuesday, Sept 25)

2018-09-27 Thread Derrick Stolee
On 9/27/2018 11:14 AM, Ben Peart wrote: On 9/26/2018 2:44 PM, Derrick Stolee wrote: On 9/26/2018 1:59 PM, Junio C Hamano wrote: Derrick Stolee writes: Perhaps removing the middle columns of data and just " ) " would be easier? We could also remove tabs to save space. Fo

Re: Git Test Coverage Report (Tuesday, Sept 25)

2018-09-27 Thread Derrick Stolee
On 9/27/2018 11:21 AM, Ben Peart wrote: On 9/26/2018 2:54 PM, Derrick Stolee wrote: GIT_TEST_INDEX_THREADS=1 Because the test repos are so small (ie smaller than 10K files), the test suite already executes as if GIT_TEST_INDEX_THREADS=1 is set (ie single threaded).  I added the test

Re: Git for Windows for Unix?

2018-09-27 Thread Derrick Stolee
On 9/27/2018 12:01 PM, Ævar Arnfjörð Bjarmason wrote: I had an IRC conversation with Johannes saying I didn't know Git For Windows builds perfectly well for Linux, this just isn't advertised in the ANNOUNCE E-Mails, so I hadn't tried. We run CI to ensure it builds and tests on Mac OSX, too.

Re: [PATCH v3 7/7] revision.c: refactor basic topo-order logic

2018-09-27 Thread Derrick Stolee
On 9/21/2018 1:39 PM, Derrick Stolee via GitGitGadget wrote: From: Derrick Stolee When running a command like 'git rev-list --topo-order HEAD', Git performed the following steps: 1. Run limit_list(), which parses all reachable commits, adds them to a linked list, and distributes

Re: [PATCH v2 0/4] git-commit-graph.txt: various cleanups

2018-09-27 Thread Derrick Stolee
and looks good to me. Reviewed-by: Derrick Stolee

Re: [RFC PATCH] We should add a "git gc --auto" after "git clone" due to commit graph

2018-10-05 Thread Derrick Stolee
On 10/5/2018 3:47 PM, Jeff King wrote: On Fri, Oct 05, 2018 at 03:41:40PM -0400, Derrick Stolee wrote: So can we really just take (total_objects - commit_graph_objects) and compare it to some threshold? The commit-graph only stores the number of _commits_, not total objects. Oh, right

Re: [RFC PATCH] We should add a "git gc --auto" after "git clone" due to commit graph

2018-10-05 Thread Derrick Stolee
On 10/5/2018 3:21 PM, Jeff King wrote: On Fri, Oct 05, 2018 at 09:45:47AM -0400, Derrick Stolee wrote: My misunderstanding was that your proposed change to gc computes the commit-graph in either of these two cases: (1) The auto-GC threshold is met. (2) There is no commit-graph file

Re: [PATCH 1/1] commit-graph: define GIT_TEST_COMMIT_GRAPH

2018-10-08 Thread Derrick Stolee
On 10/8/2018 10:58 AM, Ævar Arnfjörð Bjarmason wrote: On Mon, Oct 08 2018, Derrick Stolee wrote: On 10/8/2018 9:43 AM, Ævar Arnfjörð Bjarmason wrote: On Tue, Aug 28 2018, Derrick Stolee via GitGitGadget wrote: From: Derrick Stolee The commit-graph feature is tested in isolation by t5318

Re: [PATCH 1/1] commit-graph: define GIT_TEST_COMMIT_GRAPH

2018-10-08 Thread Derrick Stolee
On 10/8/2018 9:43 AM, Ævar Arnfjörð Bjarmason wrote: On Tue, Aug 28 2018, Derrick Stolee via GitGitGadget wrote: From: Derrick Stolee The commit-graph feature is tested in isolation by t5318-commit-graph.sh and t6600-test-reach.sh, but there are many more interesting scenarios involving

Re: We should add a "git gc --auto" after "git clone" due to commit graph

2018-10-08 Thread Derrick Stolee
On 10/8/2018 2:10 PM, SZEDER Gábor wrote: On Mon, Oct 08, 2018 at 12:57:34PM -0400, Derrick Stolee wrote: Nice! These numbers make sense to me, in terms of how many TREESAME queries we actually need to perform for such a query. Yeah... because you didn't notice that I deliberately cheated

Re: We should add a "git gc --auto" after "git clone" due to commit graph

2018-10-08 Thread Derrick Stolee
On 10/8/2018 12:41 PM, SZEDER Gábor wrote: On Wed, Oct 03, 2018 at 03:18:05PM -0400, Jeff King wrote: I'm still excited about the prospect of a bloom filter for paths which each commit touches. I think that's the next big frontier in getting things like "git log -- path" to a reasonable

Re: [PATCH][Outreachy] remove all the inclusions of git-compat-util.h in header files

2018-10-08 Thread Derrick Stolee
On 10/8/2018 1:05 PM, Ananya Krishna Maram wrote: Hi All, Hello, Ananya! Welcome. I was searching through #leftovers and found this. https://public-inbox.org/git/cabpp-bgvvxcbzx44er6to-pusfen_6gnyj1u5cuon9deaa4...@mail.gmail.com/ This patch address the task discussed in the above link. The

Re: [PATCH v3 7/7] revision.c: refactor basic topo-order logic

2018-10-11 Thread Derrick Stolee
On 10/11/2018 11:35 AM, Jeff King wrote: On Fri, Sep 21, 2018 at 10:39:36AM -0700, Derrick Stolee via GitGitGadget wrote: From: Derrick Stolee When running a command like 'git rev-list --topo-order HEAD', Git performed the following steps: [...] In the new algorithm, these three steps

Re: [PATCH v3 1/2] commit-graph write: add progress output

2018-10-12 Thread Derrick Stolee
On 10/12/2018 11:07 AM, Ævar Arnfjörð Bjarmason wrote: On Fri, Oct 12 2018, Junio C Hamano wrote: Makes sense. If this second iteration were also time consuming, then it probably is a good idea to split these into two separate phases? "Counting 1...N" followed by "Inspecting 1...N" or

Re: [PATCH v2 0/3] Add GIT_TEST_MULTI_PACK_INDEX environment variable

2018-10-12 Thread Derrick Stolee
On 10/12/2018 1:34 PM, Derrick Stolee via GitGitGadget wrote: To increase coverage of the multi-pack-index feature, add a GIT_TEST_MULTI_PACK_INDEX environment variable similar to other GIT_TEST_* variables. After creating the environment variable and running the test suite with it enabled, I

Re: [PATCH v4 0/1] contrib: Add script to show uncovered "new" lines

2018-10-12 Thread Derrick Stolee
On 10/11/2018 11:01 PM, Junio C Hamano wrote: "Derrick Stolee via GitGitGadget" writes: CHANGES IN V4: I reduced the blame output using -s which decreases the width. I include a summary of the commit authors at the end to help people see the lines they wrote. This version is a

Re: Git Test Coverage Report (Monday, Oct 15)

2018-10-15 Thread Derrick Stolee
On 10/15/2018 12:24 PM, Derrick Stolee wrote: Uncovered code in 'jch' (22f2f0f) and not in 'next' (152ad8e) - prio-queue.c 2d181390f3 94) return queue->array[queue->nr - 1].data; (I have a fix to cover this in my private

Re: Git Test Coverage Report (Friday, Oct 12)

2018-10-15 Thread Derrick Stolee
On 10/15/2018 4:09 AM, Johannes Schindelin wrote: Hi Stolee, On Fri, 12 Oct 2018, Derrick Stolee wrote: In an effort to ensure new code is reasonably covered by the test suite, we now have contrib/coverage-diff.sh to combine the gcov output from 'make coverage-test ; make coverage-report

Re: [PATCH v2 09/13] commit-graph: convert to using the_hash_algo

2018-10-15 Thread Derrick Stolee
On 10/14/2018 10:18 PM, brian m. carlson wrote: Instead of using hard-coded constants for object sizes, use the_hash_algo to look them up. In addition, use a function call to look up the object ID version and produce the correct value. This looks good and I can see already how this new

Re: [PATCH 0/4] Bloom filter experiment

2018-10-15 Thread Derrick Stolee
On 10/9/2018 3:34 PM, SZEDER Gábor wrote: To keep the ball rolling, here is my proof of concept in a somewhat cleaned-up form, with still plenty of rough edges. Peff, Szeder, and Jonathan, Thanks for giving me the kick in the pants to finally write a proof of concept for my personal take on

Re: [PATCH v2 13/13] commit-graph: specify OID version for SHA-256

2018-10-15 Thread Derrick Stolee
On 10/14/2018 10:19 PM, brian m. carlson wrote: Since the commit-graph code wants to serialize the hash algorithm into the data store, specify a version number for each supported algorithm. Note that we don't use the values of the constants themselves, as they are internal and could change in

Re: [PATCH 0/4] Bloom filter experiment

2018-10-16 Thread Derrick Stolee
On 10/16/2018 12:45 AM, Junio C Hamano wrote: Derrick Stolee writes: 2. The filters are sized according to the number of changes in each commit, with a minimum of one 64-bit word. ... 6. When we compute the Bloom filters, we don't store a filter for commits whose first-parent diff has more

Re: commit-graph is cool (overcoming add_missing_tags() perf issues)

2018-10-17 Thread Derrick Stolee
On 10/17/2018 2:00 PM, Elijah Newren wrote: Hi, Just wanted to give a shout-out for the commit-graph work and how impressive it is. I had an internal report from a user that git pushes containing only one new tiny commit were taking over a minute (in a moderate size repo with good network

Re: [PATCH] multi-pack-index: avoid dead store for struct progress

2018-10-18 Thread Derrick Stolee
On 10/18/2018 2:59 PM, Carlo Marcelo Arenas Belón wrote: it is initialized unconditionally by a call to start_progress below. Signed-off-by: Carlo Marcelo Arenas Belón --- midx.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/midx.c b/midx.c index ea2f3ffe2e..4fac0cd08a

Re: [PATCH v4 4/7] revision.c: begin refactoring --topo-order logic

2018-10-21 Thread Derrick Stolee
On 10/21/2018 9:12 PM, Junio C Hamano wrote: Jakub Narebski writes: So if revs->limited is set (but not because revs->topo_order is set), which means A..B queries, we will be still using the old algorithm. All right, though I wonder if it could be improved in the future (perhaps with the help

[RFC PATCH] revision.c: use new algorithm in A..B case

2018-10-21 Thread Derrick Stolee
Signed-off-by: Derrick Stolee --- I just wanted to mention that in order to use the new logic for 'git log --topo-order A..B', we just need the following patch. It is an extra time that sets 'revs->limited' to 1, triggering the old logic. You can use this for comparison purposes, but

Re: What's cooking in git.git (Oct 2018, #04; Fri, 19)

2018-10-19 Thread Derrick Stolee
On 10/19/2018 2:02 AM, Junio C Hamano wrote: * ds/ci-commit-graph-and-midx (2018-10-19) 1 commit - ci: add optional test variables One of our CI tests to run with "unusual/experimental/random" settings now also uses commti-graph and midx. Will merge to 'next'.

Re: [PATCH v4 3/7] test-reach: add rev-list tests

2018-10-21 Thread Derrick Stolee
On 10/21/2018 6:21 AM, Jakub Narebski wrote: "Derrick Stolee via GitGitGadget" writes: From: Derrick Stolee The rev-list command is critical to Git's functionality. Ensure it works in the three commit-graph environments constructed in t6600-test-reach.sh. Here are a few impor

Re: [PATCH 1/3] t6501: use --quiet when testing gc stderr

2018-10-18 Thread Derrick Stolee
On 10/18/2018 1:23 AM, Junio C Hamano wrote: "Derrick Stolee via GitGitGadget" writes: From: Derrick Stolee The test script t6501-freshen-objects.sh has some tests that care if 'git gc' has any output to stderr. This is intended to say that no warnings occurred related to br

Re: [PATCH 0/3] Use commit-graph by default

2018-10-18 Thread Derrick Stolee
On 10/17/2018 11:47 PM, Junio C Hamano wrote: If I recall correctly, one more task that was discussed but hasn't been addressed well is how the generation and incremental update of it should integrate with the normal repository maintenance workflow (perhaps "gc --auto"). If we are going to turn

Re: [PATCH v2 13/13] commit-graph: specify OID version for SHA-256

2018-10-18 Thread Derrick Stolee
On 10/17/2018 8:06 PM, brian m. carlson wrote: On Wed, Oct 17, 2018 at 04:31:19PM +0200, Duy Nguyen wrote: On Wed, Oct 17, 2018 at 12:44 AM brian m. carlson wrote: Honestly, anything in the .git directory that is not the v3 pack indexes or the loose object file should be in exactly one hash

Git Test Coverage Report (Monday, Oct 15)

2018-10-15 Thread Derrick Stolee
(_list); b97e187364 4804) return -1; b97e187364 4808) return error(_("could not copy '%s' to '%s'."), todo_file, b97e187364 4812) return error(_("could not transform the todo list")); b97e187364 4841) return error(_("could not transform the todo list")); b97e187364 4844)

Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear

2018-10-15 Thread Derrick Stolee
On 10/14/2018 10:29 AM, René Scharfe wrote: It still has some repetition, converted code is a bit longer than the current one, and I don't know how to build a Coccinelle rule that would do that conversion. Looked for a possibility to at least leave QSORT call-sites alone by enhancing that

Re: [PATCH v3 4/7] revision.c: begin refactoring --topo-order logic

2018-10-12 Thread Derrick Stolee
On 10/12/2018 2:33 AM, Junio C Hamano wrote: "Derrick Stolee via GitGitGadget" writes: * revs->limited implies we run limit_list() to walk the entire reachable set. There are some short-cuts here, such as if we perform a range query like 'git rev-list COMPARE..HEAD' and w

Git Test Coverage Report (Friday, Oct 12)

2018-10-12 Thread Derrick Stolee
3 314) ce_flags = ce->ce_flags; 568f3a6073 315) base_flags = base->ce_flags; 568f3a6073 317) ce->ce_flags   &= ondisk_flags; 568f3a6073 318) base->ce_flags &= ondisk_flags; 568f3a6073 319) ret = memcmp(>ce_stat_data, >ce_stat_data, 568f3a6073 322) ce->ce_flags = ce_flags; 568f3

Re: [PATCH 0/4] Bloom filter experiment

2018-10-16 Thread Derrick Stolee
On 10/16/2018 8:57 AM, Ævar Arnfjörð Bjarmason wrote: On Tue, Oct 16 2018, Derrick Stolee wrote: On 10/16/2018 12:45 AM, Junio C Hamano wrote: Derrick Stolee writes: 2. The filters are sized according to the number of changes in each commit, with a minimum of one 64-bit word. ... 6. When

Re: [PATCH v2 13/13] commit-graph: specify OID version for SHA-256

2018-10-17 Thread Derrick Stolee
On 10/14/2018 10:19 PM, brian m. carlson wrote: Since the commit-graph code wants to serialize the hash algorithm into the data store, specify a version number for each supported algorithm. Note that we don't use the values of the constants themselves, as they are internal and could change in

Re: [PATCH 00/19] Bring more repository handles into our code base

2018-10-17 Thread Derrick Stolee
On 10/16/2018 7:35 PM, Stefan Beller wrote: This series takes another approach as it doesn't change the signature of functions, but introduces new functions that can deal with arbitrary repositories, keeping the old function signature around using a shallow wrapper.

Re: [PATCH] test-tool: show tool list on error

2018-10-17 Thread Derrick Stolee
utors' lives! Thanks. Reviewed-by: Derrick Stolee

Re: [PATCH 0/1] Run GIT_TEST_COMMIT_GRAPH and GIT_TEST_MULTI_PACK_INDEX during CI

2018-10-17 Thread Derrick Stolee
On 10/17/2018 9:00 AM, Derrick Stolee via GitGitGadget wrote: [1] https://git.visualstudio.com/git/_build?definitionId=4 Newlines are hard. Sorry for the formatting issues when translating from a PR description. Build definition that tests Git with different arrangements of GIT_TEST_

Git Test Coverage Report (Wednesday, Oct 17)

2018-10-17 Thread Derrick Stolee
gc_log_path); 3029970275 builtin/gc.c 470) ret = error_errno(_("cannot read '%s'"), gc_log_path); fec2ed2187 builtin/gc.c 495) die(FAILED_RUN, pack_refs_cmd.argv[0]); fec2ed2187 builtin/gc.c 498) die(FAILED_RUN, reflog.argv[0]); 3029970275 builtin/gc.c 585) exit(128); fec2ed2187 buil

Re: [PATCH v2 13/13] commit-graph: specify OID version for SHA-256

2018-10-16 Thread Derrick Stolee
On 10/16/2018 11:35 AM, Duy Nguyen wrote: On Mon, Oct 15, 2018 at 4:23 AM brian m. carlson wrote: Since the commit-graph code wants to serialize the hash algorithm into the data store, specify a version number for each supported algorithm. Note that we don't use the values of the constants

Re: [PATCH v3 09/12] commit-graph: convert to using the_hash_algo

2018-10-24 Thread Derrick Stolee
On 10/21/2018 10:43 PM, brian m. carlson wrote: Instead of using hard-coded constants for object sizes, use the_hash_algo to look them up. In addition, use a function call to look up the object ID version and produce the correct value. For now, we use version 1, which means to use the default

Re: [PATCH] commit-reach: fix sorting commits by generation

2018-10-24 Thread Derrick Stolee
On 10/23/2018 4:32 PM, Thomas Gummerer wrote: On 10/22, René Scharfe wrote: Am 22.10.2018 um 23:10 schrieb Thomas Gummerer: Anyway, your implied question was discussed back then. Derrick wrote: The reason to sort is to hopefully minimize the amount we walk by exploring the "lower"

Recommended configurations (was Re: [PATCH v1 2/2] reset: add new reset.quietDefault config setting)

2018-10-24 Thread Derrick Stolee
On 10/23/2018 4:03 PM, Ævar Arnfjörð Bjarmason wrote: [snip] The --ahead-behind config setting stalled on-list before: https://public-inbox.org/git/36e3a9c3-f7e2-4100-1bfc-647b809a0...@jeffhostetler.com/ Now we have this similarly themed thing. I think we need to be mindful of how changes like

Re: [PATCH v4 6/7] revision.c: generation-based topo-order algorithm

2018-10-23 Thread Derrick Stolee
On 10/22/2018 9:37 AM, Jakub Narebski wrote: "Derrick Stolee via GitGitGadget" writes: From: Derrick Stolee The current --topo-order algorithm requires walking all reachable commits up front, topo-sorting them, all before outputting the first value. This patch introduces a new

Re: [RFC] Generation Number v2

2018-10-29 Thread Derrick Stolee
On 10/29/2018 3:22 PM, Stefan Beller wrote: Based on the performance results alone, we should remove minimum generation numbers, (epoch, date) pairs, and FELINE index from consideration. There are enough examples of these indexes performing poorly. In contrast, maximum generation numbers and

Re: [RFC] Generation Number v2

2018-10-29 Thread Derrick Stolee
Here is a re-formatted version of the tables I introduced earlier. The tables were too wide for public-inbox to render correctly (when paired with my email client). Hopefully this bulleted-list format works better. Thanks, Stefan, for pointing out the rendering problems! ### Test 1: `git log

Re: [PATCH] packfile: close multi-pack-index in close_all_packs

2018-10-29 Thread Derrick Stolee
On 10/29/2018 7:10 AM, SZEDER Gábor wrote: On Thu, Oct 25, 2018 at 12:54:05PM +, Derrick Stolee wrote: Whenever we delete pack-files from the object directory, we need to also delete the multi-pack-index that may refer to those objects. Sometimes, this connection is obvious, like during

Re: [RFC] Generation Number v2

2018-10-31 Thread Derrick Stolee
On 10/29/2018 11:59 PM, Junio C Hamano wrote: Derrick Stolee writes: **V3: Corrected Commit Date.** For a commit C, let its _corrected commit date_ (denoted by cdate(C)) be the maximum of the commit date of C and the commit dates of its parents. "maximum of the commit date

Re: [PATCH v3 4/8] merge-recursive: new function for better colliding conflict resolutions

2018-10-31 Thread Derrick Stolee
On 10/19/2018 3:31 PM, Elijah Newren wrote: +#if 0 // #if-0-ing avoids unused function warning; will make live in next commit +static int handle_file_collision(struct merge_options *o, +const char *collide_path, +const char

Re: [PATCH v3 8/8] merge-recursive: improve rename/rename(1to2)/add[/add] handling

2018-10-31 Thread Derrick Stolee
On 10/19/2018 3:31 PM, Elijah Newren wrote: [snip] + char *new_path = NULL; + if (dir_in_way(b->path, !o->call_depth, 0)) { + new_path = unique_path(o, b->path, ci->branch2); + output(o, 1,

Re: [PATCH v3 2/8] t6036, t6042: testcases for rename collision of already conflicting files

2018-10-31 Thread Derrick Stolee
On 10/19/2018 3:31 PM, Elijah Newren wrote: +test_expect_success "setup nested conflicts" ' nit: should these test names be single-quoted? I see you using double-quotes in PATCH 1/8 as well, but that seems to be because there are variables in the test names. ... +test_expect_failure "check

Re: [PATCH 18/19] submodule: use submodule repos for object lookup

2018-10-31 Thread Derrick Stolee
On 10/16/2018 7:35 PM, Stefan Beller wrote: @@ -482,14 +483,46 @@ void prepare_submodule_repo_env(struct argv_array *out) DEFAULT_GIT_DIR_ENVIRONMENT); } -/* Helper function to display the submodule header line prior to the full - * summary output. If it can locate

Re: [PATCH v3 4/8] merge-recursive: new function for better colliding conflict resolutions

2018-10-31 Thread Derrick Stolee
On 10/31/2018 9:53 AM, Derrick Stolee wrote: On 10/19/2018 3:31 PM, Elijah Newren wrote: +#if 0 // #if-0-ing avoids unused function warning; will make live in next commit +static int handle_file_collision(struct merge_options *o, + const char *collide_path

Re: [RFC] Generation Number v2

2018-10-31 Thread Derrick Stolee
On 10/31/2018 8:54 AM, Ævar Arnfjörð Bjarmason wrote: On Tue, Oct 30 2018, Junio C Hamano wrote: Derrick Stolee writes: In contrast, maximum generation numbers and corrected commit dates both performed quite well. They are frequently the top two performing indexes, and rarely significantly

Re: [PATCH 0/3] Make add_missing_tags() linear

2018-11-01 Thread Derrick Stolee
On 11/1/2018 2:52 AM, Elijah Newren wrote: On Wed, Oct 31, 2018 at 5:05 AM Derrick Stolee wrote: On 10/31/2018 2:04 AM, Elijah Newren wrote: On the original repo where the topic was brought up, with commit-graph NOT turned on and using origin/master, I see: $ time git push --dry-run

Re: [PATCH v4 0/7] Use generation numbers for --topo-order

2018-11-01 Thread Derrick Stolee
On 11/1/2018 1:21 AM, Junio C Hamano wrote: "Derrick Stolee via GitGitGadget" writes: This patch series performs a decently-sized refactoring of the revision-walk machinery. Well, "refactoring" is probably the wrong word, as I don't actually remove the old code. Instead,

[PATCH v5 5/7] commit/revisions: bookkeeping before refactoring

2018-11-01 Thread Derrick Stolee
King Signed-off-by: Derrick Stolee --- commit.c | 9 - commit.h | 7 +++ revision.c | 18 ++ 3 files changed, 21 insertions(+), 13 deletions(-) diff --git a/commit.c b/commit.c index d0f199e122..a025a0db60 100644 --- a/commit.c +++ b/commit.c @@ -655,11 +655,10

[PATCH v5 4/7] revision.c: begin refactoring --topo-order logic

2018-11-01 Thread Derrick Stolee
the old logic. While this commit presents method redirection for performing the exact same logic as before, it allows the next commit to focus only on the new logic. Signed-off-by: Derrick Stolee --- revision.c | 42 ++ revision.h | 4 2 files changed

[PATCH v5 7/7] t6012: make rev-list tests more interesting

2018-11-01 Thread Derrick Stolee
numbers. The extra tests can be added indepently. Signed-off-by: Derrick Stolee --- t/t6012-rev-list-simplify.sh | 45 1 file changed, 36 insertions(+), 9 deletions(-) diff --git a/t/t6012-rev-list-simplify.sh b/t/t6012-rev-list-simplify.sh index b5a1190ffe

[PATCH v5 6/7] revision.c: generation-based topo-order algorithm

2018-11-01 Thread Derrick Stolee
0 hits). However, get the benefit that the output is presented to the user as it is discovered, much the same as a normal 'git log' command (no '--topo-order'). This is an improved user experience, even if the command has the same runtime. Helped-by: Jeff King Signed-off-by: Derrick Stolee ---

[PATCH v5 2/7] test-reach: add run_three_modes method

2018-11-01 Thread Derrick Stolee
to be a simple translation on a more general run_three_modes method that executes the given command and tests the actual output to the expected output. Signed-off-by: Derrick Stolee --- t/t6600-test-reach.sh | 12 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/t/t6600-test-reach.sh

[PATCH v5 3/7] test-reach: add rev-list tests

2018-11-01 Thread Derrick Stolee
* Ancestry: git rev-list --topo-order --ancestry-path compare..HEAD * Symmetric Difference: git rev-list --topo-order compare...HEAD Signed-off-by: Derrick Stolee --- t/t6600-test-reach.sh | 84 +++ 1 file changed, 84 insertions(+) diff --git a/t/t6600-test

[PATCH v5 1/7] prio-queue: add 'peek' operation

2018-11-01 Thread Derrick Stolee
this method is exercised by t0009-prio-queue.sh. Further, add a test that checks the behavior when the compare function is NULL (i.e. the queue becomes a stack). Signed-off-by: Derrick Stolee --- prio-queue.c | 9 + prio-queue.h | 6 ++ t/helper/test-prio

[PATCH v5 0/7] Use generation numbers for --topo-order

2018-11-01 Thread Derrick Stolee
I'm not submitting this version via GitGitGadget because it's currently struggling with how to handle a PR in a conflict state. The new flags in revision.h have a conflict with recent changes in master. Derrick Stolee (7): prio-queue: add 'peek' operation test-reach: add run_three_modes met

Re: [RFC] Generation Number v2

2018-11-01 Thread Derrick Stolee
On 11/1/2018 8:27 AM, Jakub Narebski wrote: [I have noticed that in some places I wrote A..B instead of B..A. Sorry about that] Derrick Stolee writes: Please also let me know about any additional tests that I could run. Now that I've got a lot of test scripts built up, I can re-run

master updated? (was Re: What's cooking in git.git (Nov 2018, #01; Thu, 1))

2018-11-01 Thread Derrick Stolee
On 11/1/2018 5:59 AM, Junio C Hamano wrote: -- [Graduated to "master"] I see that several topics graduated, but it appears the master branch was not updated at https://github.com/gister/git. Was this intentional? I noticed because the

Re: [PATCH v5 6/7] revision.c: generation-based topo-order algorithm

2018-11-01 Thread Derrick Stolee
On 11/1/2018 11:48 AM, SZEDER Gábor wrote: On Thu, Nov 01, 2018 at 01:46:22PM +, Derrick Stolee wrote: 1. EXPLORE: using the explore_queue priority queue (ordered by maximizing the generation number) 2. INDEGREE: using the indegree_queue priority queue (ordered by maximizing

Git Test Coverage Report (Thursday, Nov 1) Part A

2018-11-01 Thread Derrick Stolee
h; bcd33ec25f  742) name_i = error(_("'GIT_AUTHOR_NAME' already given")); bcd33ec25f  747) email_i = error(_("'GIT_AUTHOR_EMAIL' already given")); bcd33ec25f  752) date_i = error(_("'GIT_AUTHOR_DATE' already given")); bcd33ec25f  756) err = error(_("unknown v

Re: [PATCH 1/3] commit-reach: implement get_reachable_subset

2018-10-31 Thread Derrick Stolee
On 10/31/2018 2:07 AM, Elijah Newren wrote: On Tue, Oct 30, 2018 at 7:16 AM Derrick Stolee via GitGitGadget wrote: --- a/commit-reach.c +++ b/commit-reach.c @@ -688,3 +688,73 @@ int can_all_from_reach(struct commit_list *from, struct commit_list *to, object_array_clear(_objs

Re: [PATCH 1/3] commit-reach: implement get_reachable_subset

2018-10-31 Thread Derrick Stolee
On 10/30/2018 11:35 PM, Junio C Hamano wrote: "Derrick Stolee via GitGitGadget" writes: +struct commit_list *get_reachable_subset(struct commit **from, int nr_from, +struct commit **to, int nr_to, +

Re: [PATCH 0/3] Make add_missing_tags() linear

2018-10-31 Thread Derrick Stolee
On 10/31/2018 2:04 AM, Elijah Newren wrote: > On Tue, Oct 30, 2018 at 7:16 AM Derrick Stolee via GitGitGadget > wrote: >> >> As reported earlier [1], the add_missing_tags() method in remote.c has >> quadratic performance. Some of that performance is curbed due to the >

Re: commit-graph is cool (overcoming add_missing_tags() perf issues)

2018-10-30 Thread Derrick Stolee
On 10/17/2018 2:00 PM, Elijah Newren wrote: Hi, Just wanted to give a shout-out for the commit-graph work and how impressive it is. I had an internal report from a user that git pushes containing only one new tiny commit were taking over a minute (in a moderate size repo with good network

Git Test Coverage Report (Tuesday, Oct 30)

2018-10-30 Thread Derrick Stolee
introducing uncovered code: Ævar Arnfjörð Bjarmason  879b6a9e6: i18n: remote.c: mark error(...) messages for translation Antonio Ospite  45f5ef3d7: submodule: factor out a config_set_in_gitmodules_file_gently function Antonio Ospite  bcbc780d1: submodule: add a print_config_from_gitmodules() helper Anton

Re: [PATCH v3 7/7] revision.c: refactor basic topo-order logic

2018-10-25 Thread Derrick Stolee
On 10/25/2018 5:43 AM, Jeff King wrote: On Thu, Oct 11, 2018 at 12:21:44PM -0400, Derrick Stolee wrote: 2. INDEGREE: using the indegree_queue priority queue (ordered by maximizing the generation number), add one to the in- degree of each parent for each commit that is walked. Since

[PATCH] packfile: close multi-pack-index in close_all_packs

2018-10-25 Thread Derrick Stolee
failure when running t6500-gc.sh with GIT_TEST_MULTI_PACK_INDEX=1. Reported-by: Szeder Gábor Signed-off-by: Derrick Stolee --- Thanks for the report, Szeder! I think this is the correct fix, and more likely to be permanent across all builtins that run auto-GC. I based it on ds/test-multi-pack-index so

Git Test Coverage Report (Saturday, Oct 27)

2018-10-27 Thread Derrick Stolee
o_list() to work on a todo_list Alban Gruin  b8dac44d1: sequencer: refactor skip_unnecessary_picks() to work on a todo_list Antonio Ospite  45f5ef3d7: submodule: factor out a config_set_in_gitmodules_file_gently function Antonio Ospite  bcbc780d1: submodule: add a print_config_from_gitmodules() hel

Git Test Coverage Report (Friday, Nov 2)

2018-11-02 Thread Derrick Stolee
NAME' already given")); bcd33ec25f  747) email_i = error(_("'GIT_AUTHOR_EMAIL' already given")); bcd33ec25f  752) date_i = error(_("'GIT_AUTHOR_DATE' already given")); bcd33ec25f  756) err = error(_("unknown variable '%s'"), bcd33ec25f  761) error(_("missing 'GIT

Re: [PATCH 0/3] Make add_missing_tags() linear

2018-11-01 Thread Derrick Stolee
On 11/1/2018 2:57 PM, Elijah Newren wrote: On Thu, Nov 1, 2018 at 5:32 AM Derrick Stolee wrote: No rush. I'd just like to understand how removing the commit-graph file can make the new algorithm faster. Putting a similar count in the old algorithm would involve giving a count for every call

Re: [PATCH 19/24] submodule: use submodule repos for object lookup

2018-11-02 Thread Derrick Stolee
On 10/30/2018 6:08 PM, Stefan Beller wrote: This converts the 'show_submodule_header' function to use the repository API properly, such that the submodule objects are not added to the main object store. Signed-off-by: Stefan Beller A couple tests are broken in 'pu' when run with

Re: [PATCH v3 8/8] merge-recursive: improve rename/rename(1to2)/add[/add] handling

2018-11-02 Thread Derrick Stolee
On 11/2/2018 1:27 PM, Elijah Newren wrote: On Thu, Nov 1, 2018 at 12:01 AM Elijah Newren wrote: On Wed, Oct 31, 2018 at 8:08 AM Derrick Stolee wrote: On 10/19/2018 3:31 PM, Elijah Newren wrote: [snip] + char *new_path = NULL; + if (dir_in_way(b->p

Re: [PATCH 19/24] submodule: use submodule repos for object lookup

2018-11-02 Thread Derrick Stolee
On 11/2/2018 1:23 PM, Stefan Beller wrote: On Fri, Nov 2, 2018 at 6:03 AM Derrick Stolee wrote: On 10/30/2018 6:08 PM, Stefan Beller wrote: This converts the 'show_submodule_header' function to use the repository API properly, such that the submodule objects are not added to the main object

Re: [PATCH 0/3] Make add_missing_tags() linear

2018-11-02 Thread Derrick Stolee
On 11/2/2018 10:58 AM, Elijah Newren wrote: On Thu, Nov 1, 2018 at 12:02 PM Derrick Stolee wrote: On 11/1/2018 2:57 PM, Elijah Newren wrote: On Thu, Nov 1, 2018 at 5:32 AM Derrick Stolee wrote: No rush. I'd just like to understand how removing the commit-graph file can make the new

[PATCH] merge-recursive: combine error handling

2018-11-02 Thread Derrick Stolee
, and this output is duplicated for each side of the conflict. Simplify the call by generating this new path in a helper function. Signed-off-by: Derrick Stolee --- Elijah, Here is a patch that combines the logic to avoid code clones, and also more easily covers code blocks. Of course, your

Re: [PATCH v4 00/10] Improve path collision conflict resolutions

2018-11-02 Thread Derrick Stolee
On 11/2/2018 2:53 PM, Elijah Newren wrote: Major question: * You'll note that I edited the last two patches to mark them as RFC. To be honest, I'm not sure what to do with these. They improve code coverage of new code, but the same gaps existed in the old code; they only show

[RFC] Generation Number v2

2018-10-29 Thread Derrick Stolee
We've discussed in several places how to improve upon generation numbers. This RFC is a report based on my investigation into a few new options, and how they compare for Git's purposes on several existing open-source repos. You can find this report and the associated test scripts at

Re: Git Test Coverage Report (Saturday, Oct 27)

2018-10-27 Thread Derrick Stolee
On 10/27/2018 9:55 AM, Junio C Hamano wrote: Derrick Stolee writes: Uncovered in mater not in master@{1} Does this typo indicate that some part of the process to produce and send out this report involve manual editing? I kick off four builds

<    6   7   8   9   10   11   12   13   14   >