Re: [PATCH on sb/more-repo-in-api] revision: use commit graph in get_reference()

2018-12-07 Thread Derrick Stolee
On 12/6/2018 6:36 PM, Jonathan Tan wrote: AFAICT oid_object_info doesn't take advantage of the commit graph, but just looks up the object header, which is still less than completely parsing it. Then lookup_commit is overly strict, as it may return NULL as when there still is a type mismatch (I

Re: [PATCH v2 2/3] commit-graph: fix buffer read-overflow

2018-12-07 Thread Derrick Stolee
On 12/6/2018 3:20 PM, Josh Steadmon wrote: + +# usage: corrupt_and_zero_graph_then_verify +# Manipulates the commit-graph file at by inserting the data, +# then zeros the file starting at . Finally, runs +# 'git commit-graph verify' and places the output in the file 'err'. Tests 'err' +#

Re: [PATCH 2/2] commit-graph: fix buffer read-overflow

2018-12-06 Thread Derrick Stolee
On 12/5/2018 5:32 PM, Josh Steadmon wrote: + if (chunk_lookup + GRAPH_CHUNKLOOKUP_WIDTH > data + graph_size) { + error(_("chunk lookup table entry missing; graph file may be incomplete")); + free(graph); + return NULL; +

Re: git, monorepos, and access control

2018-12-05 Thread Derrick Stolee
On 12/5/2018 3:34 PM, Ævar Arnfjörð Bjarmason wrote: On Wed, Dec 05 2018, Coiner, John wrote: I'm an engineer with AMD. I'm looking at whether we could switch our internal version control to a monorepo, possibly one based on git and VFSForGit. Has anyone looked at adding access control to

Git Test Coverage Report (Friday Nov 30)

2018-11-30 Thread Derrick Stolee
Here is today's test coverage report. Thanks, -Stolee [1] https://dev.azure.com/git/git/_build/results?buildId=277 --- pu: 5a1a9a96d55fbb80426189a921d7b6cc66564c78 jch: 71c29cabb7379fe9abaacbbbd1350268d0c18b4f next: a9faaff8c120bf4783cb892c157871fe524b3608 master:

Re: [PATCH 3/5] pack-objects: add --sparse option

2018-11-30 Thread Derrick Stolee
On 11/29/2018 9:39 PM, Junio C Hamano wrote: Derrick Stolee writes: While _eventually_ we should make this opt-out, we shouldn't do that until it has cooked a while. I actually do not agree. If the knob gives enough benefit, the users will learn about it viva voce, and in a few more

[PATCH v2 5/6] pack-objects: create pack.useSparse setting

2018-11-29 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The '--sparse' flag in 'git pack-objects' changes the algorithm used to enumerate objects to one that is faster for individual users pushing new objects that change only a small cone of the working directory. The sparse algorithm is not recommended for a server, which likely

[PATCH v2 0/6] Add a new "sparse" tree walk algorithm

2018-11-29 Thread Derrick Stolee via GitGitGadget
V2: * Added GIT_TEST_PACK_SPARSE test option. * Fixed test breakages when GIT_TEST_PACK_SPARSE is enabled by adding null checks. Derrick Stolee (6): revision: add mark_tree_uninteresting_sparse list-objects: consume sparse tree walk pack-objects: add --sparse option revision: i

[PATCH v2 1/6] revision: add mark_tree_uninteresting_sparse

2018-11-29 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee In preparation for a new algorithm that walks fewer trees when creating a pack from a set of revisions, create a method that takes an oidset of tree oids and marks reachable objects as UNINTERESTING. The current implementation uses the existing mark_tree_uninteresting

[PATCH v2 2/6] list-objects: consume sparse tree walk

2018-11-29 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When creating a pack-file using 'git pack-objects --revs' we provide a list of interesting and uninteresting commits. For example, a push operation would make the local topic branch be interesting and the known remote refs as uninteresting. We want to discover the set of new

[PATCH v2 6/6] pack-objects: create GIT_TEST_PACK_SPARSE

2018-11-29 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Create a test variable GIT_TEST_PACK_SPARSE to enable the sparse object walk algorithm by default during the test suite. Enabling this variable ensures coverage in many interesting cases, such as shallow clones, partial clones, and missing objects. Signed-off-by: Derrick

[PATCH v2 4/6] revision: implement sparse algorithm

2018-11-29 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When enumerating objects to place in a pack-file during 'git pack-objects --revs', we discover the "frontier" of commits that we care about and the boundary with commit we find uninteresting. From that point, we walk trees to discover which trees and blobs are uni

[PATCH v2 3/6] pack-objects: add --sparse option

2018-11-29 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Add a '--sparse' option flag to the pack-objects builtin. This allows the user to specify that they want to use the new logic for walking trees. This logic currently does not differ from the existing output, but will in a later change. Create a new test script, t5322-pack

Re: [PATCH 3/5] pack-objects: add --sparse option

2018-11-29 Thread Derrick Stolee
On 11/28/2018 5:11 PM, Stefan Beller wrote: +--sparse:: + Use the "sparse" algorithm to determine which objects to include in + the pack. This can have significant performance benefits when computing + a pack to send a small change. However, it is possible that extra +

Re: [PATCH 0/5] Add a new "sparse" tree walk algorithm

2018-11-28 Thread Derrick Stolee
On 11/28/2018 5:18 PM, Ævar Arnfjörð Bjarmason wrote: This is really interesting. I tested this with: diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 124b1bafc4..5c7615f06c 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -3143

[PATCH 4/5] revision: implement sparse algorithm

2018-11-28 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When enumerating objects to place in a pack-file during 'git pack-objects --revs', we discover the "frontier" of commits that we care about and the boundary with commit we find uninteresting. From that point, we walk trees to discover which trees and blobs are uni

[PATCH 0/5] Add a new "sparse" tree walk algorithm

2018-11-28 Thread Derrick Stolee via GitGitGadget
py" because a rename would (usually) change the original folder and trigger a walk into that path, discovering the objects. In order to benefit from this approach, the user can opt-in using the pack.useSparse config setting. This setting can be overridden using the '--no-sparse' opti

[PATCH 3/5] pack-objects: add --sparse option

2018-11-28 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Add a '--sparse' option flag to the pack-objects builtin. This allows the user to specify that they want to use the new logic for walking trees. This logic currently does not differ from the existing output, but will in a later change. Create a new test script, t5322-pack

[PATCH 5/5] pack-objects: create pack.useSparse setting

2018-11-28 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The '--sparse' flag in 'git pack-objects' changes the algorithm used to enumerate objects to one that is faster for individual users pushing new objects that change only a small cone of the working directory. The sparse algorithm is not recommended for a server, which likely

[PATCH 2/5] list-objects: consume sparse tree walk

2018-11-28 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When creating a pack-file using 'git pack-objects --revs' we provide a list of interesting and uninteresting commits. For example, a push operation would make the local topic branch be interesting and the known remote refs as uninteresting. We want to discover the set of new

[PATCH 1/5] revision: add mark_tree_uninteresting_sparse

2018-11-28 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee In preparation for a new algorithm that walks fewer trees when creating a pack from a set of revisions, create a method that takes an oidset of tree oids and marks reachable objects as UNINTERESTING. The current implementation uses the existing mark_tree_uninteresting

Re: [BUG REPORT] t5322: demonstrate a pack-objects bug

2018-11-28 Thread Derrick Stolee
On 11/28/2018 2:45 PM, Derrick Stolee wrote: I was preparing a new "sparse" algorithm for calculating the interesting objects to send on push. The important steps happen during 'git pack-objects', so I was creating test cases to see how the behavior changes in narrow cases. Specific

[BUG REPORT] t5322: demonstrate a pack-objects bug

2018-11-28 Thread Derrick Stolee
s probable that this is not a minimal test case, but happens to be the test I had created before discovering the problem. I compiled v2.17.0 and v2.12.0 as checks to see if I could find a "good" commit with which to start a bisect, but both failed. This is an old bug! Signed-off-by: Derrick Stole

Git Test Coverage Report (Wednesday Nov 21)

2018-11-21 Thread Derrick Stolee
error_errno(_("unable to dup bundle descriptor")); 2c8ee1f53c 268) child_process_clear(_objects); 2c8ee1f53c 269) return -1; 2c8ee1f53c 478) rollback_lock_file(); config.c fast-import.c ca473cef91 2958) strbuf_addf(, "%s %s %"PRIuMAX"\n", oid_to_hex(oid), midx.c na

Re: [PATCH v2 1/1] Use size_t instead of 'unsigned long' for data in memory

2018-11-21 Thread Derrick Stolee
ot;, "sz", or "len". However, it appears to be correct from context. Thanks for this! Reviewed-by: Derrick Stolee

Re: [PATCH 2/2] commit-graph: don't call write_graph_chunk_large_edges() unnecessarily

2018-11-21 Thread Derrick Stolee
On 11/20/2018 8:26 PM, SZEDER Gábor wrote: write_graph_chunk_data(f, GRAPH_OID_LEN, commits.list, commits.nr); - write_graph_chunk_large_edges(f, commits.list, commits.nr); + if (num_large_edges) + write_graph_chunk_large_edges(f, commits.list, commits.nr);

Re: [PATCH 1/2] commit-graph: rename 'num_extra_edges' variable to 'num_large_edges'

2018-11-21 Thread Derrick Stolee
On 11/20/2018 10:29 PM, Junio C Hamano wrote: SZEDER Gábor writes: I rename these variables to 'num_large_edges', because the commit graph file format speaks about the 'Large Edge List' chunk. However, I do find that the term 'extra' makes much more sense Would it make sense to do the

Re: [PATCH 1/1] revision.c: use new topo-order logic in tests

2018-11-20 Thread Derrick Stolee
On 11/20/2018 1:13 AM, Junio C Hamano wrote: "Derrick Stolee via GitGitGadget" writes: @@ -3143,6 +3144,9 @@ int prepare_revision_walk(struct rev_info *revs) commit_list_sort_by_date(>commits); if (revs->no_walk) return 0; + i

Re: [PATCH 0/3] delta-island fixes

2018-11-20 Thread Derrick Stolee
On 11/20/2018 4:44 AM, Jeff King wrote: In cases like this I think it's often a good idea to have a perf test. Those are expensive anyway, and we'd have the double benefit of exercising the code and showing off the performance improvement. But the delta-island code only makes sense on a very

Re: Git Test Coverage Report (v2.20.0-rc0)

2018-11-20 Thread Derrick Stolee
On 11/20/2018 6:34 AM, Jeff King wrote: On Mon, Nov 19, 2018 at 10:40:53AM -0500, Derrick Stolee wrote: Since depth is never incremented, we are not covering this block. Is it possible to test? This should be covered by the fix in: https://public-inbox.org/git/20181120095053.gc22

Re: Git Test Coverage Report (v2.20.0-rc0)

2018-11-19 Thread Derrick Stolee
On 11/19/2018 1:33 PM, Ævar Arnfjörð Bjarmason wrote: On Mon, Nov 19 2018, Derrick Stolee wrote: [...] builtin/rebase.c 62c23938fa 55) return env; [...] Ævar Arnfjörð Bjarmason 62c23938f: tests: add a special setup where rebase.useBuiltin is off This one would be covered

Re: Git Test Coverage Report (v2.20.0-rc0)

2018-11-19 Thread Derrick Stolee
On 11/19/2018 2:00 PM, Ben Peart wrote: On 11/19/2018 10:40 AM, Derrick Stolee wrote: There are a lot of lines introduced by the IEOT extension in these commits:  > Ben Peart  3255089ad: ieot: add Index Entry Offset Table (IEOT) extension  > Ben Peart  3b1d9e045: eoie: a

Re: [PATCH] commit-graph: split up close_reachable() progress output

2018-11-19 Thread Derrick Stolee
On 11/19/2018 3:23 PM, Ævar Arnfjörð Bjarmason wrote: + if (report_progress) + progress = start_delayed_progress( + _("Expanding reachable commits in commit graph"), j = 0); This should be the only one that shows up in all but the very largest of

Re: Git Test Coverage Report (v2.20.0-rc0)

2018-11-19 Thread Derrick Stolee
On 11/19/2018 2:39 PM, Ævar Arnfjörð Bjarmason wrote: On Mon, Nov 19 2018, Derrick Stolee wrote: The coverage report has been using the following: export GIT_TEST_MULTI_PACK_INDEX=1 export GIT_TEST_COMMIT_GRAPH=1 export GIT_TEST_INDEX_VERION=4 export GIT_TEST_SPLIT_INDEX=yes export

Re: Git Test Coverage Report (v2.20.0-rc0)

2018-11-19 Thread Derrick Stolee
On 11/19/2018 1:33 PM, Ævar Arnfjörð Bjarmason wrote: On Mon, Nov 19 2018, Derrick Stolee wrote: Here is a test coverage report for the uncovered lines introduced in v2.20.0-rc0 compared to v2.19.1. Thanks a lot for this. [...] builtin/rebase.c 62c23938fa 55) return env; [...] Ævar Arnfjörð

[PATCH 1/1] revision.c: use new topo-order logic in tests

2018-11-19 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The revision-walk machinery is being rewritten to use generation numbers in the commit-graph when availble. Due to some problematic commit histories, the new logic can be slower than the previous method due to how commit dates and generation numbers interact. Thus, the new

[PATCH 0/1] Use new topo-order logic with GIT_TEST_COMMIT_GRAPH

2018-11-19 Thread Derrick Stolee via GitGitGadget
, but that doesn't seem necessary. Thanks, -Stolee Derrick Stolee (1): revision.c: use new topo-order logic in tests revision.c | 4 1 file changed, 4 insertions(+) base-commit: 561b583749b7428f1790f03164d0d0e75be71d7b Published-As: https://github.com/gitgitgadget/git/releases/tags/pr-83

Re: Git Test Coverage Report (v2.20.0-rc0)

2018-11-19 Thread Derrick Stolee
was related to unusual error conditions (e.g. open_pack_index() fails) The comments below are intended only to point out potential directions to improve test coverage. Some of it is for me to do! Thanks, -Stolee On 11/18/2018 9:54 PM, Derrick Stolee wrote: 66ec0390e7 builtin/fsck.c 888) midx_argv[2

Git Test Coverage Report (v2.20.0-rc0)

2018-11-18 Thread Derrick Stolee
en_relative, >shallows, worktree.c 3a3b9d8cde 495) return -1; 3a3b9d8cde 508) return -1; 3a3b9d8cde 517) return -1; ab3e1f78ae 537) break; wt-status.c f3bd35fa0d  671) s->committable = 1; 73ba5d78b4 1958) if (s->state.rebase_in_progress || 73ba5d78b4 1959) s->state.rebase_interac

Re: [PATCH/RFC v1 1/1] Use size_t instead of unsigned long

2018-11-18 Thread Derrick Stolee
On 11/17/2018 10:11 AM, tbo...@web.de wrote: From: Torsten Bögershausen Currently Git users can not commit files >4Gib under 64 bit Windows, where "long" is 32 bit but size_t is 64 bit. Improve the code base in small steps, as small as possible. What started with a small patch to replace

Git Test Coverage Report (Sunday, Nov 18th)

2018-11-18 Thread Derrick Stolee
of %s"), oid_to_hex(oid)); bac2a1e36f  586) goto leave_reset_head; 3249c1251e  590) ret = error(_("failed to find tree of %s"), oid_to_hex(oid)); 3249c1251e  591) goto leave_reset_head; 3249c1251e  604) goto leave_reset_head; builtin/show-branch.c 517fe807d6 builtin/show-branch.c

Re: [PATCH] technical doc: add a design doc for the evolve command

2018-11-16 Thread Derrick Stolee
On 11/14/2018 7:55 PM, sxe...@google.com wrote: From: Stefan Xenos This document describes what an obsolescence graph for git would look like, the behavior of the evolve command, and the changes planned for other commands. Thanks for putting this together! diff --git

Re: [PATCHv3 00/23] Bring more repository handles into our code base

2018-11-16 Thread Derrick Stolee
On 11/13/2018 7:12 PM, Stefan Beller wrote: Please have a look at the last 4 patches specifically as they were new in the last iteration (but did not receive any comment), as they demonstrate and fix a problem that is only exposed when using GIT_TEST_COMMIT_GRAPH=1 for the test suite. I took a

Re: [PATCH v5 02/12] sha1-file: provide functions to look up hash algorithms

2018-11-13 Thread Derrick Stolee
On 11/4/2018 6:44 PM, brian m. carlson wrote: +int hash_algo_by_name(const char *name) +{ + int i; + if (!name) + return GIT_HASH_UNKNOWN; + for (i = 1; i < GIT_HASH_NALGOS; i++) + if (!strcmp(name, hash_algos[i].name)) + return

Git Test Coverage Report (Tuesday, Nov 13)

2018-11-13 Thread Derrick Stolee
die(_("unable to access commit %s"), builtin/show-branch.c 517fe807d6 builtin/show-branch.c 607) BUG_ON_OPT_NEG(unset); builtin/show-ref.c 517fe807d6 builtin/show-ref.c 154) BUG_ON_OPT_NEG(unset); builtin/upload-archive.c e001fd3a50 builtin/upload-archive.c 113) if (version == protocol_v0 || ve

Re: [PATCH 0/9] caching loose objects

2018-11-12 Thread Derrick Stolee
On 11/12/2018 9:46 AM, Jeff King wrote: Here's the series I mentioned earlier in the thread to cache loose objects when answering has_object_file(..., OBJECT_INFO_QUICK). For those just joining us, this makes operations that look up a lot of missing objects (like "index-pack" looking for

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-11-12 Thread Derrick Stolee
On 11/12/2018 9:54 AM, Jeff King wrote: In cases where we expect to ask has_sha1_file() about a lot of objects that we are not likely to have (e.g., during fetch negotiation), we already use OBJECT_INFO_QUICK to sacrifice accuracy (due to racing with a simultaneous write or repack) for speed (we

Re: [PATCH 5/9] handle alternates paths the same as the main object dir

2018-11-12 Thread Derrick Stolee
On 11/12/2018 10:46 AM, Jeff King wrote: On Mon, Nov 12, 2018 at 10:38:28AM -0500, Derrick Stolee wrote: We could go either direction here, but this patch moves the alternates struct over to the main directory style (rather than vice-versa). Technically the alternates style is more

Re: [PATCH 6/9] sha1-file: use an object_directory for the main object dir

2018-11-12 Thread Derrick Stolee
On 11/12/2018 9:50 AM, Jeff King wrote: Our handling of alternate object directories is needlessly different from the main object directory. As a result, many places in the code basically look like this: do_something(r->objects->objdir); for (odb = r->objects->alt_odb_list; odb; odb =

Re: [PATCH 5/9] handle alternates paths the same as the main object dir

2018-11-12 Thread Derrick Stolee
On 11/12/2018 9:49 AM, Jeff King wrote: When we generate loose file paths for the main object directory, the caller provides a buffer to loose_object_path (formerly sha1_file_name). The callers generally keep their own static buffer to avoid excessive reallocations. But for alternate

Re: [PATCH 4/9] sha1_file_name(): overwrite buffer instead of appending

2018-11-12 Thread Derrick Stolee
On 11/12/2018 9:48 AM, Jeff King wrote: Since we're changing the semantics, let's take the opportunity to give it a more hash-neutral name (which will also catch any callers from topics in flight). THANK YOU! This method name confused me so much when I was first looking at the code, but the

Re: [PATCH 3/9] rename "alternate_object_database" to "object_directory"

2018-11-12 Thread Derrick Stolee
On 11/12/2018 9:48 AM, Jeff King wrote: In preparation for unifying the handling of alt odb's and the normal repo object directory, let's use a more neutral name. This patch is purely mechanical, swapping the type name, and converting any variables named "alt" to "odb". There should be no

Re: [PATCH 1/9] fsck: do not reuse child_process structs

2018-11-12 Thread Derrick Stolee
On 11/12/2018 9:46 AM, Jeff King wrote: The run-command API makes no promises about what is left in a struct child_process after a command finishes, and it's not safe to simply reuse it again for a similar command. In particular: - if you use child->args or child->env_array, they are cleared

Git Test Coverage Report (Wednesday, Nov 7)

2018-11-07 Thread Derrick Stolee
UNINTERESTING; b45424181e 2954) return; b45424181e 2957) mark_parents_uninteresting(c); b45424181e 2980) return; b45424181e 2983) return; b45424181e 3048) continue; f0d9cc4196 3097) if (!revs->ignore_missing_links) f0d9cc4196 3098) die("Failed to traverse parents of commit %s", f0d9cc419

[PATCH v2 1/1] pack-objects: ignore ambiguous object warnings

2018-11-06 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee A git push process runs several processes during its run, but one includes git send-pack which calls git pack-objects and passes the known have/wants into stdin using object ids. However, the default setting for core.warnAmbiguousRefs requires git pack-objects to check

[PATCH v2 0/1] send-pack: set core.warnAmbiguousRefs=false

2018-11-06 Thread Derrick Stolee via GitGitGadget
use anyone viewing the message without threading. Derrick Stolee (1): pack-objects: ignore ambiguous object warnings builtin/pack-objects.c | 6 ++ 1 file changed, 6 insertions(+) base-commit: cae598d9980661a978e2df4fb338518f7bf09572 Published-As: https://github.com/gitgitgadget/git/relea

Re: [PATCH 0/1] send-pack: set core.warnAmbiguousRefs=false

2018-11-06 Thread Derrick Stolee
On 11/6/2018 2:51 PM, Jeff King wrote: On Tue, Nov 06, 2018 at 02:44:42PM -0500, Jeff King wrote: The fix for this is simple: set core.warnAmbiguousRefs to false for this specific call of git pack-objects coming from git send-pack. We don't want to default it to false for all calls to git

Re: [PATCH 0/1] send-pack: set core.warnAmbiguousRefs=false

2018-11-06 Thread Derrick Stolee
On 11/6/2018 2:44 PM, Jeff King wrote: On Tue, Nov 06, 2018 at 11:13:47AM -0800, Derrick Stolee via GitGitGadget wrote: I've been looking into the performance of git push for very large repos. Our users are reporting that 60-80% of git push time is spent during the "Enumerating objects&q

[PATCH 1/1] send-pack: set core.warnAmbiguousRefs=false

2018-11-06 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee During a 'git push' command, we run 'git send-pack' inside of our transport helper. This creates a 'git pack-objects' process and passes a list of object ids. If this list is large, then the pack-objects process can spend a lot of time checking the possible refs

[PATCH 0/1] send-pack: set core.warnAmbiguousRefs=false

2018-11-06 Thread Derrick Stolee via GitGitGadget
github.com/Microsoft/git/pull/67 Derrick Stolee (1): send-pack: set core.warnAmbiguousRefs=false send-pack.c | 2 ++ 1 file changed, 2 insertions(+) base-commit: cae598d9980661a978e2df4fb338518f7bf09572 Published-As: https://github.com/gitgitgadget/git/releases/tags/pr-68%2Fderrickstolee%2

Re: [PATCH] multi-pack-index: make code -Wunused-parameter clean

2018-11-05 Thread Derrick Stolee
se nr_objects to make sure we don't walk too far. Signed-off-by: Jeff King Thanks, both, for catching this. I prefer the approach that adds defenses. Reviewed-by: Derrick Stolee --- midx.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/midx.c b/midx.c ind

Git Test Coverage Report (Friday, Nov 2)

2018-11-02 Thread Derrick Stolee
NAME' already given")); bcd33ec25f  747) email_i = error(_("'GIT_AUTHOR_EMAIL' already given")); bcd33ec25f  752) date_i = error(_("'GIT_AUTHOR_DATE' already given")); bcd33ec25f  756) err = error(_("unknown variable '%s'"), bcd33ec25f  761) error(_("missing 'GIT

[PATCH] merge-recursive: combine error handling

2018-11-02 Thread Derrick Stolee
, and this output is duplicated for each side of the conflict. Simplify the call by generating this new path in a helper function. Signed-off-by: Derrick Stolee --- Elijah, Here is a patch that combines the logic to avoid code clones, and also more easily covers code blocks. Of course, your

Re: [PATCH v4 00/10] Improve path collision conflict resolutions

2018-11-02 Thread Derrick Stolee
On 11/2/2018 2:53 PM, Elijah Newren wrote: Major question: * You'll note that I edited the last two patches to mark them as RFC. To be honest, I'm not sure what to do with these. They improve code coverage of new code, but the same gaps existed in the old code; they only show

Re: [PATCH v3 8/8] merge-recursive: improve rename/rename(1to2)/add[/add] handling

2018-11-02 Thread Derrick Stolee
On 11/2/2018 1:27 PM, Elijah Newren wrote: On Thu, Nov 1, 2018 at 12:01 AM Elijah Newren wrote: On Wed, Oct 31, 2018 at 8:08 AM Derrick Stolee wrote: On 10/19/2018 3:31 PM, Elijah Newren wrote: [snip] + char *new_path = NULL; + if (dir_in_way(b->p

Re: [PATCH 19/24] submodule: use submodule repos for object lookup

2018-11-02 Thread Derrick Stolee
On 11/2/2018 1:23 PM, Stefan Beller wrote: On Fri, Nov 2, 2018 at 6:03 AM Derrick Stolee wrote: On 10/30/2018 6:08 PM, Stefan Beller wrote: This converts the 'show_submodule_header' function to use the repository API properly, such that the submodule objects are not added to the main object

Re: [PATCH 0/3] Make add_missing_tags() linear

2018-11-02 Thread Derrick Stolee
On 11/2/2018 10:58 AM, Elijah Newren wrote: On Thu, Nov 1, 2018 at 12:02 PM Derrick Stolee wrote: On 11/1/2018 2:57 PM, Elijah Newren wrote: On Thu, Nov 1, 2018 at 5:32 AM Derrick Stolee wrote: No rush. I'd just like to understand how removing the commit-graph file can make the new

[PATCH v2 0/3] Make add_missing_tags() linear

2018-11-02 Thread Derrick Stolee via GitGitGadget
(). Thanks, -Stolee [1] https://public-inbox.org/git/cabpp-becpsoxudovjbdg_3w9wus102rw+e+qpmd4g3qyd-q...@mail.gmail.com/ Derrick Stolee (3): commit-reach: implement get_reachable_subset test-reach: test get_reachable_subset remote: make add_missing_tags() linear commit-reach.c| 70

[PATCH v2 1/3] commit-reach: implement get_reachable_subset

2018-11-02 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The existing reachability algorithms in commit-reach.c focus on finding merge-bases or determining if all commits in a set X can reach at least one commit in a set Y. However, for two commits sets X and Y, we may also care about which commits in Y are reachable from at least

[PATCH v2 3/3] remote: make add_missing_tags() linear

2018-11-02 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The add_missing_tags() method currently has quadratic behavior. This is due to a linear number (based on number of tags T) of calls to in_merge_bases_many, which has linear performance (based on number of commits C in the repository). Replace this O(T * C) algorithm

[PATCH v2 2/3] test-reach: test get_reachable_subset

2018-11-02 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The get_reachable_subset() method returns the list of commits in the 'to' array that are reachable from at least one commit in the 'from' array. Add tests that check this method works in a few cases: 1. All commits in the 'to' list are reachable. This exercises the early

Re: [PATCH 19/24] submodule: use submodule repos for object lookup

2018-11-02 Thread Derrick Stolee
On 10/30/2018 6:08 PM, Stefan Beller wrote: This converts the 'show_submodule_header' function to use the repository API properly, such that the submodule objects are not added to the main object store. Signed-off-by: Stefan Beller A couple tests are broken in 'pu' when run with

Re: [PATCH 0/3] Make add_missing_tags() linear

2018-11-01 Thread Derrick Stolee
On 11/1/2018 2:57 PM, Elijah Newren wrote: On Thu, Nov 1, 2018 at 5:32 AM Derrick Stolee wrote: No rush. I'd just like to understand how removing the commit-graph file can make the new algorithm faster. Putting a similar count in the old algorithm would involve giving a count for every call

Git Test Coverage Report (Thursday, Nov 1) Part A

2018-11-01 Thread Derrick Stolee
h; bcd33ec25f  742) name_i = error(_("'GIT_AUTHOR_NAME' already given")); bcd33ec25f  747) email_i = error(_("'GIT_AUTHOR_EMAIL' already given")); bcd33ec25f  752) date_i = error(_("'GIT_AUTHOR_DATE' already given")); bcd33ec25f  756) err = error(_("unknown v

Re: [PATCH v5 6/7] revision.c: generation-based topo-order algorithm

2018-11-01 Thread Derrick Stolee
On 11/1/2018 11:48 AM, SZEDER Gábor wrote: On Thu, Nov 01, 2018 at 01:46:22PM +, Derrick Stolee wrote: 1. EXPLORE: using the explore_queue priority queue (ordered by maximizing the generation number) 2. INDEGREE: using the indegree_queue priority queue (ordered by maximizing

master updated? (was Re: What's cooking in git.git (Nov 2018, #01; Thu, 1))

2018-11-01 Thread Derrick Stolee
On 11/1/2018 5:59 AM, Junio C Hamano wrote: -- [Graduated to "master"] I see that several topics graduated, but it appears the master branch was not updated at https://github.com/gister/git. Was this intentional? I noticed because the

Re: [PATCH v4 0/7] Use generation numbers for --topo-order

2018-11-01 Thread Derrick Stolee
On 11/1/2018 1:21 AM, Junio C Hamano wrote: "Derrick Stolee via GitGitGadget" writes: This patch series performs a decently-sized refactoring of the revision-walk machinery. Well, "refactoring" is probably the wrong word, as I don't actually remove the old code. Instead,

[PATCH v5 6/7] revision.c: generation-based topo-order algorithm

2018-11-01 Thread Derrick Stolee
0 hits). However, get the benefit that the output is presented to the user as it is discovered, much the same as a normal 'git log' command (no '--topo-order'). This is an improved user experience, even if the command has the same runtime. Helped-by: Jeff King Signed-off-by: Derrick Stolee ---

[PATCH v5 2/7] test-reach: add run_three_modes method

2018-11-01 Thread Derrick Stolee
to be a simple translation on a more general run_three_modes method that executes the given command and tests the actual output to the expected output. Signed-off-by: Derrick Stolee --- t/t6600-test-reach.sh | 12 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/t/t6600-test-reach.sh

[PATCH v5 3/7] test-reach: add rev-list tests

2018-11-01 Thread Derrick Stolee
* Ancestry: git rev-list --topo-order --ancestry-path compare..HEAD * Symmetric Difference: git rev-list --topo-order compare...HEAD Signed-off-by: Derrick Stolee --- t/t6600-test-reach.sh | 84 +++ 1 file changed, 84 insertions(+) diff --git a/t/t6600-test

[PATCH v5 5/7] commit/revisions: bookkeeping before refactoring

2018-11-01 Thread Derrick Stolee
King Signed-off-by: Derrick Stolee --- commit.c | 9 - commit.h | 7 +++ revision.c | 18 ++ 3 files changed, 21 insertions(+), 13 deletions(-) diff --git a/commit.c b/commit.c index d0f199e122..a025a0db60 100644 --- a/commit.c +++ b/commit.c @@ -655,11 +655,10

[PATCH v5 4/7] revision.c: begin refactoring --topo-order logic

2018-11-01 Thread Derrick Stolee
the old logic. While this commit presents method redirection for performing the exact same logic as before, it allows the next commit to focus only on the new logic. Signed-off-by: Derrick Stolee --- revision.c | 42 ++ revision.h | 4 2 files changed

[PATCH v5 7/7] t6012: make rev-list tests more interesting

2018-11-01 Thread Derrick Stolee
numbers. The extra tests can be added indepently. Signed-off-by: Derrick Stolee --- t/t6012-rev-list-simplify.sh | 45 1 file changed, 36 insertions(+), 9 deletions(-) diff --git a/t/t6012-rev-list-simplify.sh b/t/t6012-rev-list-simplify.sh index b5a1190ffe

[PATCH v5 0/7] Use generation numbers for --topo-order

2018-11-01 Thread Derrick Stolee
I'm not submitting this version via GitGitGadget because it's currently struggling with how to handle a PR in a conflict state. The new flags in revision.h have a conflict with recent changes in master. Derrick Stolee (7): prio-queue: add 'peek' operation test-reach: add run_three_modes met

[PATCH v5 1/7] prio-queue: add 'peek' operation

2018-11-01 Thread Derrick Stolee
this method is exercised by t0009-prio-queue.sh. Further, add a test that checks the behavior when the compare function is NULL (i.e. the queue becomes a stack). Signed-off-by: Derrick Stolee --- prio-queue.c | 9 + prio-queue.h | 6 ++ t/helper/test-prio

Re: [RFC] Generation Number v2

2018-11-01 Thread Derrick Stolee
On 11/1/2018 8:27 AM, Jakub Narebski wrote: [I have noticed that in some places I wrote A..B instead of B..A. Sorry about that] Derrick Stolee writes: Please also let me know about any additional tests that I could run. Now that I've got a lot of test scripts built up, I can re-run

Re: [PATCH 0/3] Make add_missing_tags() linear

2018-11-01 Thread Derrick Stolee
On 11/1/2018 2:52 AM, Elijah Newren wrote: On Wed, Oct 31, 2018 at 5:05 AM Derrick Stolee wrote: On 10/31/2018 2:04 AM, Elijah Newren wrote: On the original repo where the topic was brought up, with commit-graph NOT turned on and using origin/master, I see: $ time git push --dry-run

Re: [PATCH v3 8/8] merge-recursive: improve rename/rename(1to2)/add[/add] handling

2018-10-31 Thread Derrick Stolee
On 10/19/2018 3:31 PM, Elijah Newren wrote: [snip] + char *new_path = NULL; + if (dir_in_way(b->path, !o->call_depth, 0)) { + new_path = unique_path(o, b->path, ci->branch2); + output(o, 1,

Re: [PATCH v3 2/8] t6036, t6042: testcases for rename collision of already conflicting files

2018-10-31 Thread Derrick Stolee
On 10/19/2018 3:31 PM, Elijah Newren wrote: +test_expect_success "setup nested conflicts" ' nit: should these test names be single-quoted? I see you using double-quotes in PATCH 1/8 as well, but that seems to be because there are variables in the test names. ... +test_expect_failure "check

Re: [PATCH v3 4/8] merge-recursive: new function for better colliding conflict resolutions

2018-10-31 Thread Derrick Stolee
On 10/31/2018 9:53 AM, Derrick Stolee wrote: On 10/19/2018 3:31 PM, Elijah Newren wrote: +#if 0 // #if-0-ing avoids unused function warning; will make live in next commit +static int handle_file_collision(struct merge_options *o, + const char *collide_path

Re: [PATCH v3 4/8] merge-recursive: new function for better colliding conflict resolutions

2018-10-31 Thread Derrick Stolee
On 10/19/2018 3:31 PM, Elijah Newren wrote: +#if 0 // #if-0-ing avoids unused function warning; will make live in next commit +static int handle_file_collision(struct merge_options *o, +const char *collide_path, +const char

Re: [PATCH 18/19] submodule: use submodule repos for object lookup

2018-10-31 Thread Derrick Stolee
On 10/16/2018 7:35 PM, Stefan Beller wrote: @@ -482,14 +483,46 @@ void prepare_submodule_repo_env(struct argv_array *out) DEFAULT_GIT_DIR_ENVIRONMENT); } -/* Helper function to display the submodule header line prior to the full - * summary output. If it can locate

Re: [RFC] Generation Number v2

2018-10-31 Thread Derrick Stolee
On 10/31/2018 8:54 AM, Ævar Arnfjörð Bjarmason wrote: On Tue, Oct 30 2018, Junio C Hamano wrote: Derrick Stolee writes: In contrast, maximum generation numbers and corrected commit dates both performed quite well. They are frequently the top two performing indexes, and rarely significantly

Re: [RFC] Generation Number v2

2018-10-31 Thread Derrick Stolee
On 10/29/2018 11:59 PM, Junio C Hamano wrote: Derrick Stolee writes: **V3: Corrected Commit Date.** For a commit C, let its _corrected commit date_ (denoted by cdate(C)) be the maximum of the commit date of C and the commit dates of its parents. "maximum of the commit date

Re: [PATCH 0/3] Make add_missing_tags() linear

2018-10-31 Thread Derrick Stolee
On 10/31/2018 2:04 AM, Elijah Newren wrote: > On Tue, Oct 30, 2018 at 7:16 AM Derrick Stolee via GitGitGadget > wrote: >> >> As reported earlier [1], the add_missing_tags() method in remote.c has >> quadratic performance. Some of that performance is curbed due to the >

Re: [PATCH 1/3] commit-reach: implement get_reachable_subset

2018-10-31 Thread Derrick Stolee
On 10/30/2018 11:35 PM, Junio C Hamano wrote: "Derrick Stolee via GitGitGadget" writes: +struct commit_list *get_reachable_subset(struct commit **from, int nr_from, +struct commit **to, int nr_to, +

Re: [PATCH 1/3] commit-reach: implement get_reachable_subset

2018-10-31 Thread Derrick Stolee
On 10/31/2018 2:07 AM, Elijah Newren wrote: On Tue, Oct 30, 2018 at 7:16 AM Derrick Stolee via GitGitGadget wrote: --- a/commit-reach.c +++ b/commit-reach.c @@ -688,3 +688,73 @@ int can_all_from_reach(struct commit_list *from, struct commit_list *to, object_array_clear(_objs

Re: commit-graph is cool (overcoming add_missing_tags() perf issues)

2018-10-30 Thread Derrick Stolee
On 10/17/2018 2:00 PM, Elijah Newren wrote: Hi, Just wanted to give a shout-out for the commit-graph work and how impressive it is. I had an internal report from a user that git pushes containing only one new tiny commit were taking over a minute (in a moderate size repo with good network

[PATCH 2/3] test-reach: test get_reachable_subset

2018-10-30 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The get_reachable_subset() method returns the list of commits in the 'to' array that are reachable from at least one commit in the 'from' array. Add tests that check this method works in a few cases: 1. All commits in the 'to' list are reachable. This exercises the early

[PATCH 3/3] remote: make add_missing_tags() linear

2018-10-30 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The add_missing_tags() method currently has quadratic behavior. This is due to a linear number (based on number of tags T) of calls to in_merge_bases_many, which has linear performance (based on number of commits C in the repository). Replace this O(T * C) algorithm

  1   2   3   4   5   6   7   8   9   10   >