Re: [PATCH 1/2] commit-graph write: add progress output

2018-09-07 Thread Derrick Stolee
On 9/7/2018 8:40 AM, Ævar Arnfjörð Bjarmason wrote: On Tue, Sep 04 2018, Ævar Arnfjörð Bjarmason wrote: Before this change the "commit-graph write" command didn't report any progress. On my machine this command takes more than 10 seconds to write the graph for linux.git, and around 1m30s on

Re: [PATCH 09/11] multi-pack-index: verify object offsets

2018-09-07 Thread Derrick Stolee
On 9/6/2018 8:34 PM, Eric Sunshine wrote: On Wed, Sep 5, 2018 at 10:46 AM Derrick Stolee via GitGitGadget wrote: Replace the BUG() statement with a die() statement, now that we may hit a bad pack-int-id during a 'verify' command on a corrupt multi-pack-index, and it is covered by a test

[RFC PATCH 6/6] commit-reach: fix first-parent heuristic

2018-09-06 Thread Derrick Stolee
4 s (num_walked: 8,601) Large Case: 4fbcca4e~1: 24.0 s 4fbcca4e: 0.12 s (num_walked: 503,925) HEAD: 0.06 s (num_walked: 217,243) Signed-off-by: Derrick Stolee --- commit-reach.c| 4 +++- t/t6600-test-reach.sh | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/

[RFC PATCH 3/6] commit-reach: use trace2 in can_all_from_reach

2018-09-06 Thread Derrick Stolee
Signed-off-by: Derrick Stolee --- commit-reach.c | 8 1 file changed, 8 insertions(+) diff --git a/commit-reach.c b/commit-reach.c index 0fc3b1ac18..0a75644653 100644 --- a/commit-reach.c +++ b/commit-reach.c @@ -563,6 +563,7 @@ int can_all_from_reach_with_flag(struct object_array

[RFC PATCH 2/6] comit-reach: use trace2 for commit_contains_tag_algo

2018-09-06 Thread Derrick Stolee
Signed-off-by: Derrick Stolee --- commit-reach.c | 12 +++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/commit-reach.c b/commit-reach.c index ee374dce20..0fc3b1ac18 100644 --- a/commit-reach.c +++ b/commit-reach.c @@ -481,6 +481,7 @@ static enum contains_result

[RFC PATCH 4/6] test-tool: start trace2 environment

2018-09-06 Thread Derrick Stolee
Signed-off-by: Derrick Stolee --- t/helper/test-tool.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c index 7566b0786a..f70d5d74f8 100644 --- a/t/helper/test-tool.c +++ b/t/helper/test-tool.c @@ -1,5 +1,6 @@ #include "git-compat-u

[RFC PATCH 5/6] test-lib: add run_and_check_trace2

2018-09-06 Thread Derrick Stolee
with GIT_TR2_PERFORMANCE running. Then, check the output for the expected category-key-value triple. Use this function in t6600-test-reach.sh to verify can_all_from_reach only visits 11 commits in the example. Signed-off-by: Derrick Stolee --- t/t6600-test-reach.sh | 6 ++ t/test-lib.sh | 14

[RFC PATCH 1/6] commit-reach: add trace2 telemetry and walk count

2018-09-06 Thread Derrick Stolee
times we execute the while loop, which is consistent between runs. Signed-off-by: Derrick Stolee --- commit-reach.c | 8 1 file changed, 8 insertions(+) diff --git a/commit-reach.c b/commit-reach.c index 86715c103c..ee374dce20 100644 --- a/commit-reach.c +++ b/commit-reach.c @@ -38,6

[RFC PATCH 0/6] Use trace2 in commit-reach

2018-09-06 Thread Derrick Stolee
nstrated by the test suite when I fix the bug. Thanks, -Stolee P.S. I'm sending this RFC from gmail because I'm having SMTP issues with my work email. Derrick Stolee (6): commit-reach: add trace2 telemetry and walk count comit-reach: use trace2 for commit_contains_tag_algo comm

Re: [PATCH 1/2] commit-graph write: add progress output

2018-09-05 Thread Derrick Stolee
On 9/5/2018 5:46 PM, Junio C Hamano wrote: Derrick Stolee writes: for (i = 0; i < commits->nr; i++) { + display_progress(progress, i); if (commits->list[i]->generation != GENERATION_NUMBER_INFINITY && commits-

Re: [PATCH 01/11] multi-pack-index: add 'verify' verb

2018-09-05 Thread Derrick Stolee
On 9/5/2018 2:59 PM, Eric Sunshine wrote: On Wed, Sep 5, 2018 at 10:46 AM Derrick Stolee via GitGitGadget wrote: The multi-pack-index builtin writes multi-pack-index files, and uses a 'write' verb to do so. Add a 'verify' verb that checks this file matches the contents of the pack-indexes

Re: [PATCH 04/11] multi-pack-index: verify packname order

2018-09-05 Thread Derrick Stolee
On 9/5/2018 3:14 PM, Stefan Beller wrote: On Wed, Sep 5, 2018 at 12:11 PM Derrick Stolee wrote: On 9/5/2018 2:15 PM, Stefan Beller wrote: On Wed, Sep 5, 2018 at 7:46 AM Derrick Stolee via GitGitGadget wrote: From: Derrick Stolee The final check we make while loading a multi-pack-index

Re: [PATCH 04/11] multi-pack-index: verify packname order

2018-09-05 Thread Derrick Stolee
On 9/5/2018 2:15 PM, Stefan Beller wrote: On Wed, Sep 5, 2018 at 7:46 AM Derrick Stolee via GitGitGadget wrote: From: Derrick Stolee The final check we make while loading a multi-pack-index is that the packfile names are in lexicographical order. Make this error be a die() instead. What

[PATCH 11/11] fsck: verify multi-pack-index

2018-09-05 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When core.multiPackIndex is true, we may have a multi-pack-index in our object directory. Add calls to 'git multi-pack-index verify' at the end of 'git fsck' if so. Signed-off-by: Derrick Stolee --- builtin/fsck.c | 18 ++ t/t5319-multi-pack

[PATCH 10/11] multi-pack-index: report progress during 'verify'

2018-09-05 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When verifying a multi-pack-index, the only action that takes significant time is checking the object offsets. For example, to verify a multi-pack-index containing 6.2 million objects in the Linux kernel repository takes 1.3 seconds on my machine. 99% of that time is spent

[PATCH 03/11] multi-pack-index: verify corrupt chunk lookup table

2018-09-05 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Signed-off-by: Derrick Stolee --- midx.c | 3 +++ t/t5319-multi-pack-index.sh | 13 + 2 files changed, 16 insertions(+) diff --git a/midx.c b/midx.c index ec78254bb6..8b054b39ab 100644 --- a/midx.c +++ b/midx.c @@ -100,6 +100,9 @@ struct

[PATCH 07/11] multi-pack-index: verify oid lookup order

2018-09-05 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Signed-off-by: Derrick Stolee --- midx.c | 11 +++ t/t5319-multi-pack-index.sh | 8 2 files changed, 19 insertions(+) diff --git a/midx.c b/midx.c index dfd26b4d74..06d5cfc826 100644 --- a/midx.c +++ b/midx.c @@ -959,5 +959,16 @@ int

[PATCH 09/11] multi-pack-index: verify object offsets

2018-09-05 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The 'git multi-pack-index verify' command must verify the object offsets stored in the multi-pack-index are correct. There are two ways the offset chunk can be incorrect: the pack-int-id and the object offset. Replace the BUG() statement with a die() statement, now that we

[PATCH 08/11] multi-pack-index: fix 32-bit vs 64-bit size check

2018-09-05 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When loading a 64-bit offset, we intend to check that off_t can store the resulting offset. However, the condition accidentally checks the 32-bit offset to see if it is smaller than a 64-bit value. Fix it, and this will be covered by a test in the 'git multi-pack-index

[PATCH 06/11] multi-pack-index: verify oid fanout order

2018-09-05 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Signed-off-by: Derrick Stolee --- midx.c | 9 + t/t5319-multi-pack-index.sh | 8 2 files changed, 17 insertions(+) diff --git a/midx.c b/midx.c index a02b19efc1..dfd26b4d74 100644 --- a/midx.c +++ b/midx.c @@ -950,5 +950,14 @@ int

[PATCH 05/11] multi-pack-index: verify missing pack

2018-09-05 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Signed-off-by: Derrick Stolee --- midx.c | 16 t/t5319-multi-pack-index.sh | 5 + 2 files changed, 21 insertions(+) diff --git a/midx.c b/midx.c index e655a15aed..a02b19efc1 100644 --- a/midx.c +++ b/midx.c @@ -926,13 +926,29

[PATCH 04/11] multi-pack-index: verify packname order

2018-09-05 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The final check we make while loading a multi-pack-index is that the packfile names are in lexicographical order. Make this error be a die() instead. In order to test this condition, we need multiple packfiles. Earlier in t5319-multi-pack-index.sh, we tested the interaction

[PATCH 01/11] multi-pack-index: add 'verify' verb

2018-09-05 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The multi-pack-index builtin writes multi-pack-index files, and uses a 'write' verb to do so. Add a 'verify' verb that checks this file matches the contents of the pack-indexes it replaces. The current implementation is a no-op, but will be extended in small increments

[PATCH 02/11] multi-pack-index: verify bad header

2018-09-05 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When verifying if a multi-pack-index file is valid, we want the command to fail to signal an invalid file. Previously, we wrote an error to stderr and continued as if we had no multi-pack-index. Now, die() instead of error(). Add tests that check corrupted headers in a few

[PATCH 00/11] Add 'git multi-pack-index verify' command

2018-09-05 Thread Derrick Stolee via GitGitGadget
d a commit that provides progress updates when checking object offsets. Based on ds/multi-pack-index [1] https://public-inbox.org/git/20180904202729.13900-1-ava...@gmail.com/T/#u Derrick Stolee (11): multi-pack-index: add 'verify' verb multi-pack-index: verify bad header multi-pack-ind

Re: [PATCH 0/2] commit-graph: add progress output

2018-09-05 Thread Derrick Stolee
On 9/4/2018 4:27 PM, Ævar Arnfjörð Bjarmason wrote: This series adds progress output to the commit-graph command, so that when it's called by "git gc" or "git fsck" we can see what's going on with it. Ævar Arnfjörð Bjarmason (2): commit-graph write: add progress output commit-graph

Re: [PATCH 1/2] commit-graph write: add progress output

2018-09-05 Thread Derrick Stolee
On 9/4/2018 4:27 PM, Ævar Arnfjörð Bjarmason wrote: @@ -591,8 +597,13 @@ static void close_reachable(struct packed_oid_list *oids) { int i; struct commit *commit; + struct progress *progress = NULL; + int j = 0; The change below over-counts the number of commits

Re: [PATCH 1/2] commit-graph write: add progress output

2018-09-05 Thread Derrick Stolee
commit-graph write --reachable Annotating commits in commit graph: 1564087, done. Computing commit graph generation numbers: 100% (191590/191590), done. -->8-- From: Derrick Stolee Date: Wed, 5 Sep 2018 11:55:42 + Subject: [PATCH] fixup! commit-graph write: add progress output Signed-off

Re: [PATCH v2 0/1] Define GIT_TEST_COMMIT_GRAPH for commit-graph test coverage

2018-09-04 Thread Derrick Stolee
On 9/4/2018 12:49 PM, Duy Nguyen wrote: On Wed, Aug 29, 2018 at 2:49 PM Derrick Stolee via GitGitGadget wrote: The commit-graph (and multi-pack-index) features are optional data structures that can make Git operations faster. Since they are optional, we do not enable them in most Git tests

Re: [PATCH v4 12/12] t5318: use test_oid for HASH_LEN

2018-09-04 Thread Derrick Stolee
On 9/3/2018 7:25 PM, brian m. carlson wrote: From: Derrick Stolee Signed-off-by: Derrick Stolee Signed-off-by: brian m. carlson --- t/t5318-commit-graph.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index

Re: [PATCH 1/8] trace2: create new combined trace facility

2018-08-31 Thread Derrick Stolee
On 8/31/2018 12:49 PM, Jeff Hostetler via GitGitGadget wrote: + if (tr2key_trace_want(_event)) { + struct tr2tls_thread_ctx *ctx = tr2tls_get_self(); + if (ctx->nr_open_regions <= tr2env_event_depth_wanted) { This should also compare to TR2_MAX_REGION_NESTING

Re: [PATCH 0/8] WIP: trace2: a new trace facility

2018-08-31 Thread Derrick Stolee
On 8/31/2018 12:49 PM, Jeff Hostetler via GitGitGadget wrote: This patch series contains a new trace2 facility that hopefully addresses the recent trace- and structured-logging-related discussions. The intent is to eventually replace the existing trace_ routines (or to route them to the new

[PATCH 1/1] commit: don't use generation numbers if not needed

2018-08-30 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee In 3afc679b "commit: use generations in paint_down_to_common()", the queue in paint_down_to_common() was changed to use a priority order based on generation number before commit date. This served two purposes: 1. When generation numbers are present, the walk

[PATCH 0/1] v2.19.0-rc1 Performance Regression in 'git merge-base'

2018-08-30 Thread Derrick Stolee via GitGitGadget
/[PATCH 0/6] Use generation numbers for --topo-order [2] https://public-inbox.org/git/86muxcuyod@gmail.com/[RFC] Other chunks for commit-graph, part 2 - reachability indexes Cc: gitster@pobox.comCc: p...@peff.net Derrick Stolee (1): commit: don't use generation numbers if not needed

Re: Questions about the hash function transition

2018-08-29 Thread Derrick Stolee
On 8/29/2018 9:27 AM, Derrick Stolee wrote: On 8/29/2018 9:09 AM, Johannes Schindelin wrote: What I meant was to leverage the midx code, not the .midx files. My comment was motivated by my realizing that both the SHA-1 <-> SHA-256 mapping and the MIDX code have to look up (in a *fas

Re: Questions about the hash function transition

2018-08-29 Thread Derrick Stolee
On 8/29/2018 9:09 AM, Johannes Schindelin wrote: Hi Jonathan, On Tue, 28 Aug 2018, Jonathan Nieder wrote: Johannes Schindelin wrote: On Thu, 23 Aug 2018, Jonathan Nieder wrote: Ævar Arnfjörð Bjarmason wrote: Are we going to need a midx version of these mapping files? How does midx fit into

Re: [RFC PATCH 09/12] Add a base implementation of SHA-256 support

2018-08-29 Thread Derrick Stolee
On 8/28/2018 8:58 PM, brian m. carlson wrote: SHA-256 is somewhat slower to compute than SHA-1 in software. However, since our default SHA-1 implementation is collision-detecting, a reasonable cryptographic library implementation of SHA-256 will actually be faster than SHA-256. Nit: do you

Re: [RFC PATCH 08/12] commit-graph: convert to using the_hash_algo

2018-08-29 Thread Derrick Stolee
is that I predict failure in the 'git commit-graph verify' tests in t5318-commit-graph.sh. Squashing in this commit should help (assuming that test_oid works, it doesn't at my current branch): -->8-- Subject: [PATCH] t5318-commit-graph.sh: use test_oid for HASH_LEN Signed-off-by: Derr

[PATCH v2 1/1] commit-graph: define GIT_TEST_COMMIT_GRAPH

2018-08-29 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The commit-graph feature is tested in isolation by t5318-commit-graph.sh and t6600-test-reach.sh, but there are many more interesting scenarios involving commit walks. Many of these scenarios are covered by the existing test suite, but we need to maintain coverage when

[PATCH v2 0/1] Define GIT_TEST_COMMIT_GRAPH for commit-graph test coverage

2018-08-29 Thread Derrick Stolee via GitGitGadget
-NOT-MERGE: write and read commit-graph always [3] https://public-inbox.org/git/CAPig+cSjanDi=jv75pdzypajwvgd4suh3uyvy+vy7yehauy...@mail.gmail.com/ Based-On: ds/commit-graph-with-grafts Cc: jnareb@gmail.comCc: sbeller@google.comCc: sunsh...@sunshineco.com Derrick Stolee (1): commit-graph: define

Re: [PATCH v3 01/11] t: add tool to translate hash-related values

2018-08-29 Thread Derrick Stolee
On 8/28/2018 8:56 PM, brian m. carlson wrote: + rawsz="$(test_oid rawsz)" && + hexsz="$(test_oid hexsz)" && These are neat helpers! The 'git commit-graph verify' tests in t5318-commit-graph.sh should automatically work if we use these for HASH_LEN instead of 20. I'll use a

Re: [PATCH 1/1] commit-graph: define GIT_TEST_COMMIT_GRAPH

2018-08-29 Thread Derrick Stolee
On 8/28/2018 5:59 PM, Eric Sunshine wrote: On Tue, Aug 28, 2018 at 5:31 PM Derrick Stolee wrote: On 8/28/2018 4:41 PM, Stefan Beller wrote: On Tue, Aug 28, 2018 at 1:33 PM Derrick Stolee via GitGitGadget wrote: + GIT_TEST_COMMIT_GRAPH=0 && + test_must_

Re: [PATCH] commit-reach: correct accidental #include of C file

2018-08-28 Thread Derrick Stolee
: previous definition here Signed-off-by: Jonathan Nieder --- Jonathan Nieder wrote: Derrick Stolee wrote: --- a/commit-reach.c +++ b/commit-reach.c @@ -1,8 +1,10 @@ #include "cache.h" #include "commit.h" +#include "commit-graph.h" #include "decorate.h&qu

Re: [PATCH 0/9] introducing oideq()

2018-08-28 Thread Derrick Stolee
(the longer we wait, the more annoying conflicts Junio has to resolve). I'm happy to take this change whenever. In my opinion, right after v2.19.0 is cut would be a great time to merge it into master. This v2 is good. Reviewed-by: Derrick Stolee

Re: [PATCH v2 04/18] commit-reach: move commit_contains from ref-filter

2018-08-28 Thread Derrick Stolee
On 8/28/2018 5:24 PM, Jonathan Nieder wrote: Hi, Derrick Stolee wrote: There are several commit walks in the codebase. Group them together into a new commit-reach.c file and corresponding header. After we group these walks into one place, we can reduce duplicate logic by calling equivalent

Re: [PATCH 0/1] Define GIT_TEST_COMMIT_GRAPH for commit-graph test coverage

2018-08-28 Thread Derrick Stolee
On 8/28/2018 4:37 PM, Stefan Beller wrote: On Tue, Aug 28, 2018 at 1:33 PM Derrick Stolee via GitGitGadget wrote: The commit-graph (and multi-pack-index) features are optional data structures that can make Git operations faster. Since they are optional, we do not enable them in most Git tests

Re: [PATCH 1/1] commit-graph: define GIT_TEST_COMMIT_GRAPH

2018-08-28 Thread Derrick Stolee
On 8/28/2018 4:41 PM, Stefan Beller wrote: On Tue, Aug 28, 2018 at 1:33 PM Derrick Stolee via GitGitGadget wrote: From: Derrick Stolee The commit-graph feature is tested in isolation by t5318-commit-graph.sh and t6600-test-reach.sh, but there are many more interesting scenarios involving

[PATCH 1/1] commit-graph: define GIT_TEST_COMMIT_GRAPH

2018-08-28 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The commit-graph feature is tested in isolation by t5318-commit-graph.sh and t6600-test-reach.sh, but there are many more interesting scenarios involving commit walks. Many of these scenarios are covered by the existing test suite, but we need to maintain coverage when

Re: Git clean removing tracked files semi-regularly

2018-08-28 Thread Derrick Stolee
On 8/28/2018 2:29 PM, Brennan Conroy wrote: I'm using windows. Have had it happen on 8.1 and 10. "git status" shows "nothing to commit, working tree clean". I am using git version 2.17.1.windows.2 I tested on my machine and could not reproduce this issue. Could you find a full repro (first

Re: Questions about the hash function transition

2018-08-28 Thread Derrick Stolee
On 8/28/2018 8:04 AM, Johannes Schindelin wrote: Hi, On Thu, 23 Aug 2018, Jonathan Nieder wrote: Ævar Arnfjörð Bjarmason wrote: [...] Since all operations that make new objects (e.g., "git commit") add the new objects to the corresponding index, this mapping is possible for all objects in

[PATCH 1/6] prio-queue: add 'peek' operation

2018-08-27 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When consuming a priority queue, it can be convenient to inspect the next object that will be dequeued without actually dequeueing it. Our existing library did not have such a 'peek' operation, so add it as prio_queue_peek(). Add a reference-level comparison in t/helper

[PATCH 2/6] test-reach: add run_three_modes method

2018-08-27 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The 'test_three_modes' method assumes we are using the 'test-tool reach' command for our test. However, we may want to use the data shape of our commit graph and the three modes (no commit-graph, full commit-graph, partial commit-graph) for other git commands. Split

[PATCH 4/6] revision.c: begin refactoring --topo-order logic

2018-08-27 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When running 'git rev-list --topo-order' and its kin, the topo_order setting in struct rev_info implies the limited setting. This means that the following things happen during prepare_revision_walk(): * revs->limited implies we run limit_list() to walk the ent

[PATCH 3/6] test-reach: add rev-list tests

2018-08-27 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The rev-list command is critical to Git's functionality. Ensure it works in the three commit-graph environments constructed in t6600-test-reach.sh. Here are a few important types of rev-list operations: * Basic: git rev-list --topo-order HEAD * Range: git rev-list --topo

[PATCH 5/6] commit/revisions: bookkeeping before refactoring

2018-08-27 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee There are a few things that need to move around a little before making a big refactoring in the topo-order logic: 1. We need access to record_author_date() and compare_commits_by_author_date() in revision.c. These are used currently by sort_in_topological_order

[PATCH 0/6] Use generation numbers for --topo-order

2018-08-27 Thread Derrick Stolee via GitGitGadget
: Generations and Graph Algorithms [2] https://msdnshared.blob.core.windows.net/media/2018/06/commit-graph-topo-order-b-a.png Animation showing three-part walk [3] https://github.com/derrickstolee/git/tree/topo-order/testA branch containing this series along with commits to compute commit-graph in

[PATCH 6/6] revision.c: refactor basic topo-order logic

2018-08-27 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When running a command like 'git rev-list --topo-order HEAD', Git performed the following steps: 1. Run limit_list(), which parses all reachable commits, adds them to a linked list, and distributes UNINTERESTING flags. If all unprocessed commits are UNINTERESTING

Re: Contributor Summit planning

2018-08-27 Thread Derrick Stolee
On 8/27/2018 9:22 AM, Johannes Schindelin wrote: Point in favor of the pure-online meeting: the informal standup on IRC every second Friday. I really try to attend it (it is a bit awkward because it is on a Friday evening in my timezone, right at the time when I want to unwind from the work

Re: [PATCH 0/9] introducing oideq()

2018-08-27 Thread Derrick Stolee
On 8/26/2018 4:56 PM, brian m. carlson wrote: On Sat, Aug 25, 2018 at 04:00:31AM -0400, Jeff King wrote: This is a follow-up to the discussion in: https://public-inbox.org/git/20180822030344.ga14...@sigill.intra.peff.net/ The general idea is that the majority of callers don't care about

Re: [PATCH 3/9] convert "oidcmp() == 0" to oideq()

2018-08-27 Thread Derrick Stolee
On 8/27/2018 8:31 AM, Derrick Stolee wrote: On 8/25/2018 4:36 AM, Jeff King wrote: On Sat, Aug 25, 2018 at 04:07:15AM -0400, Jeff King wrote: diff --git a/contrib/coccinelle/object_id.cocci b/contrib/coccinelle/object_id.cocci index 5869979be7..548c02336d 100644 --- a/contrib/coccinelle

Re: [PATCH 3/9] convert "oidcmp() == 0" to oideq()

2018-08-27 Thread Derrick Stolee
On 8/25/2018 4:36 AM, Jeff King wrote: On Sat, Aug 25, 2018 at 04:07:15AM -0400, Jeff King wrote: diff --git a/contrib/coccinelle/object_id.cocci b/contrib/coccinelle/object_id.cocci index 5869979be7..548c02336d 100644 --- a/contrib/coccinelle/object_id.cocci +++

Re: [ANNOUNCE] Git v2.19.0-rc0

2018-08-24 Thread Derrick Stolee
On 8/23/2018 4:59 PM, Derrick Stolee wrote: When I choose my own metrics for performance tests, I like to run at least 10 runs, remove the largest AND smallest runs from the samples, and then take the average. I did this manually for 'git rev-list --all --objects' on git/git and got

Re: [PATCH v3 5/5] tests: fix and add lint for non-portable grep --file

2018-08-24 Thread Derrick Stolee
On 8/23/2018 4:44 PM, Junio C Hamano wrote: Ævar Arnfjörð Bjarmason writes: The --file option to grep isn't in POSIX[1], but -f is[1]. Let's check for that in the future, and fix the portability regression in f237c8b6fe ("commit-graph: implement git-commit-graph write", 2018-04-02) that

Re: [ANNOUNCE] Git v2.19.0-rc0

2018-08-24 Thread Derrick Stolee
On 8/24/2018 2:45 AM, Jeff King wrote: On Thu, Aug 23, 2018 at 10:59:55PM -0400, Jeff King wrote: So I think we have a winner. I'll polish that up into patches and send it out later tonight. Oof. This rabbit hole keeps going deeper and deeper. I wrote up my coccinelle findings separately in:

Re: [ANNOUNCE] Git v2.19.0-rc0

2018-08-23 Thread Derrick Stolee
On 8/23/2018 2:53 PM, Jeff King wrote: On Thu, Aug 23, 2018 at 06:26:58AM -0400, Derrick Stolee wrote: I think you can safely ignore the rest of it if you are otherwise occupied. Even if v2.19 ships without some mitigation, I don't know that it's all that big a deal, given the numbers I

[PATCH v2] config: fix commit-graph related config docs

2018-08-23 Thread Derrick Stolee
Reported-by: Szeder Gábor Signed-off-by: Derrick Stolee --- Documentation/config.txt | 17 +++-- 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/Documentation/config.txt b/Documentation/config.txt index 1c42364988..6ee1890984 100644 --- a/Documentation/config.txt +++ b/Docu

Re: [ANNOUNCE] Git v2.19.0-rc0

2018-08-23 Thread Derrick Stolee
On 8/23/2018 1:04 AM, Jeff King wrote: On Thu, Aug 23, 2018 at 03:47:07AM +, brian m. carlson wrote: I expect that's going to be the case as well. I have patches that wire up actual SHA-256 support in my hash-impl branch. However, having said that, I'm happy to defer to whatever everyone

Re: [ANNOUNCE] Git v2.19.0-rc0

2018-08-22 Thread Derrick Stolee
On 8/22/2018 12:58 PM, Duy Nguyen wrote: On Wed, Aug 22, 2018 at 6:49 PM Derrick Stolee wrote: On 8/22/2018 12:26 PM, Jeff King wrote: On Wed, Aug 22, 2018 at 06:14:24PM +0200, Duy Nguyen wrote: On Wed, Aug 22, 2018 at 6:08 PM Duy Nguyen wrote: On Wed, Aug 22, 2018 at 6:03 PM Jeff King

Re: [ANNOUNCE] Git v2.19.0-rc0

2018-08-22 Thread Derrick Stolee
On 8/22/2018 12:26 PM, Jeff King wrote: On Wed, Aug 22, 2018 at 06:14:24PM +0200, Duy Nguyen wrote: On Wed, Aug 22, 2018 at 6:08 PM Duy Nguyen wrote: On Wed, Aug 22, 2018 at 6:03 PM Jeff King wrote: On Wed, Aug 22, 2018 at 07:14:42AM -0400, Derrick Stolee wrote: The other thing I

Re: [ANNOUNCE] Git v2.19.0-rc0

2018-08-22 Thread Derrick Stolee
On 8/22/2018 1:36 AM, brian m. carlson wrote: On Tue, Aug 21, 2018 at 11:03:44PM -0400, Jeff King wrote: So I wonder if there's some other way to tell the compiler that we'll only have a few values. An enum comes to mind, though I don't think the enum rules are strict enough to make this

Re: [PATCH 3/6] t/perf: add infrastructure for measuring sizes

2018-08-22 Thread Derrick Stolee
On 8/21/2018 3:06 PM, Jeff King wrote: The main objective of scripts in the perf framework is to run "test_perf", which measures the time it takes to run some operation. However, it can also be interesting to see the change in the output size of certain operations. This patch introduces

[PATCH] config: fix commit-graph related config docs

2018-08-22 Thread Derrick Stolee
Reported-by: Szeder Gábor Signed-off-by: Derrick Stolee --- Szeder reported this in [1] and I forgot until now to come back to it. [1] https://public-inbox.org/git/20180803093909.2853-1-szeder@gmail.com/ Documentation/config.txt | 6 +- 1 file changed, 5 insertions(+), 1 del

Re: [ANNOUNCE] Git v2.19.0-rc0

2018-08-22 Thread Derrick Stolee
On 8/22/2018 3:39 AM, Ævar Arnfjörð Bjarmason wrote: On Wed, Aug 22, 2018 at 8:20 AM Jeff King wrote: On Wed, Aug 22, 2018 at 05:36:26AM +, brian m. carlson wrote: On Tue, Aug 21, 2018 at 11:03:44PM -0400, Jeff King wrote: I don't know if something like this is an improvement or now,

Re: [ANNOUNCE] Git v2.19.0-rc0

2018-08-22 Thread Derrick Stolee
On 8/21/2018 11:36 PM, Jeff King wrote: On Tue, Aug 21, 2018 at 11:03:44PM -0400, Jeff King wrote: with the obvious "oideq()" implementation added, that seems to get me to 2-3%. Not _quite_ as good as the original branching version I showed. And we had to touch all the callsites (although

Re: [PATCH v2 2/2] commit-graph.txt: improve formatting for asciidoc

2018-08-21 Thread Derrick Stolee
On 8/21/2018 5:29 PM, Junio C Hamano wrote: "Derrick Stolee via GitGitGadget" writes: From: Derrick Stolee When viewing commit-graph.txt as a plain-text document, it makes sense to keep paragraphs left-padded between bullet points. However, asciidoc converts these left-padded

[PATCH] commit: use timestamp_t for author_date_slab

2018-08-21 Thread Derrick Stolee
his would not reveal itself as a bug unless someone authors a commit after February 2106, but commits can store anything as their author date. Signed-off-by: Derrick Stolee --- I found this while reading up on the revision-walk machinery. This code hasn't been touched in years, so could apply to 'ma

Re: [ANNOUNCE] Git v2.19.0-rc0

2018-08-21 Thread Derrick Stolee
On 8/20/2018 6:13 PM, Junio C Hamano wrote: An early preview release Git v2.19.0-rc0 is now available for testing at the usual places. As part of testing the release candidate, I ran the performance suite against a fresh clone of the Linux repository using v2.18.0 and v2.19.0-rc0 (also:

Re: [PATCH 5/6] pack-bitmap: save "have" bitmap from walk

2018-08-21 Thread Derrick Stolee
On 8/21/2018 3:07 PM, Jeff King wrote: When we do a bitmap walk, we save the result, which represents (WANTs & ~HAVEs); i.e., every object we care about visiting in our walk. However, we throw away the haves bitmap, which can sometimes be useful, too. Save it and provide an access function so

Re: [PATCH] SubmittingPatches: mention doc-diff

2018-08-21 Thread Derrick Stolee
On 8/21/2018 3:23 PM, Jeff King wrote: We already advise people to make sure their documentation formats correctly. Let's point them at the doc-diff script, which can help with that. Let's also put a brief note in the script about its purpose, since that otherwise can only be found in the

[PATCH v2 2/2] commit-graph.txt: improve formatting for asciidoc

2018-08-21 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When viewing commit-graph.txt as a plain-text document, it makes sense to keep paragraphs left-padded between bullet points. However, asciidoc converts these left-padded paragraphs as monospace fonts, creating an unpleasant document. Remove the padding. The "Future

[PATCH v2 1/2] Docs: Add commit-graph tech docs to Makefile

2018-08-21 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Ensure that the commit-graph.txt and commit-graph-format.txt files are compiled to HTML using ASCIIDOC. Signed-off-by: Derrick Stolee --- Documentation/Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Documentation/Makefile b/Documentation/Makefile index

[PATCH v2 0/2] Docs: Add commit-graph tech docs to Makefile

2018-08-21 Thread Derrick Stolee via GitGitGadget
the commit messages for details. Since the files had been modified since 'maint', this version is based on 'master'. [1] https://public-inbox.org/git/20180814222846.gg142...@aiede.svl.corp.google.com/ [PATCH] partial-clone: render design doc using asciidoc Derrick Stolee (2): Docs: Add commit

Re: [PATCH v2 5/8] commit-graph: not compatible with replace objects

2018-08-21 Thread Derrick Stolee
On 8/21/2018 1:45 PM, Junio C Hamano wrote: Derrick Stolee writes: Create new method commit_graph_compatible(r) to check if a given repository r is compatible with the commit-graph feature. Fill the method with a check to see if replace-objects exist. Test this interaction succeeds, including

Re: [PATCH v2 0/8] Clarify commit-graph and grafts/replace/shallow incompatibilities

2018-08-21 Thread Derrick Stolee
On 8/21/2018 1:51 PM, Junio C Hamano wrote: Stefan Beller writes: On Mon, Aug 20, 2018 at 11:24 AM Derrick Stolee wrote: Because of the change of base, it is hard to provide a side-by-side diff from v1. I thought that was the whole point of range-diff. ;-) I thought so, too

Re: [PATCH 0/9] multi-pack-index cleanups

2018-08-21 Thread Derrick Stolee
On 8/21/2018 10:34 AM, Duy Nguyen wrote: On Mon, Aug 20, 2018 at 6:53 PM Derrick Stolee wrote: To better understand the implications of the multi-pack-index with other scenarios, I ran the test suite after adding a step to 'git repack' to write a multi-pack-index, and to default

Re: [PATCH 7/9] treewide: use get_all_packs

2018-08-21 Thread Derrick Stolee
On 8/20/2018 6:01 PM, Stefan Beller wrote: On Mon, Aug 20, 2018 at 9:52 AM Derrick Stolee wrote: There are many places in the codebase that want to iterate over all packfiles known to Git. The purposes are wide-ranging, and those that can take advantage of the multi-pack-index already do. So

Re: [PATCH 3/9] midx: mark bad packed objects

2018-08-21 Thread Derrick Stolee
On 8/20/2018 5:23 PM, Stefan Beller wrote: On Mon, Aug 20, 2018 at 9:52 AM Derrick Stolee wrote: When an object fails to decompress from a pack-file, we mark the object as 'bad' so we can retry with a different copy of the object (if such a copy exists). Before now, the multi-pack-index did

[PATCH 1/1] Docs: Add commit-graph tech docs to Makefile

2018-08-21 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Ensure that the commit-graph.txt and commit-graph-format.txt files are compiled to HTML using ASCIIDOC. Signed-off-by: Derrick Stolee --- Documentation/Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Documentation/Makefile b/Documentation/Makefile index

[PATCH 0/1] Docs: Add commit-graph tech docs to Makefile

2018-08-21 Thread Derrick Stolee via GitGitGadget
editing for format. [1] https://public-inbox.org/git/20180814222846.gg142...@aiede.svl.corp.google.com/ [PATCH] partial-clone: render design doc using asciidoc Derrick Stolee (1): Docs: Add commit-graph tech docs to Makefile Documentation/Makefile | 2 ++ 1 file changed, 2 insertions

Re: [PATCH v2 0/8] Clarify commit-graph and grafts/replace/shallow incompatibilities

2018-08-20 Thread Derrick Stolee
On 8/20/2018 3:37 PM, Stefan Beller wrote: On Mon, Aug 20, 2018 at 11:24 AM Derrick Stolee wrote: One unresolved issue with the commit-graph feature is that it can cause issues when combined with replace objects, commit grafts, or shallow clones. These are not 100% incompatible, as one could

[PATCH v2 1/8] refs.c: migrate internal ref iteration to pass thru repository argument

2018-08-20 Thread Derrick Stolee
-by: Derrick Stolee --- refs.c | 39 +-- refs.h | 10 ++ refs/iterator.c | 6 +++--- refs/refs-internal.h | 5 +++-- 4 files changed, 53 insertions(+), 7 deletions(-) diff --git a/refs.c b/refs.c index 457fb78057..7cd76f72d2

[PATCH v2 8/8] commit-graph: close_commit_graph before shallow walk

2018-08-20 Thread Derrick Stolee
ent this interaction. Signed-off-by: Derrick Stolee --- commit-graph.c | 8 commit-graph.h | 1 + upload-pack.c | 2 ++ 3 files changed, 7 insertions(+), 4 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index cee2caab5c..4bd1a4abbf 100644 --- a/commit-graph.c +++ b/commi

[PATCH v2 2/8] refs.c: upgrade for_each_replace_ref to be a each_repo_ref_fn callback

2018-08-20 Thread Derrick Stolee
From: Stefan Beller Signed-off-by: Stefan Beller Signed-off-by: Derrick Stolee --- builtin/replace.c | 8 refs.c| 9 - refs.h| 2 +- replace-object.c | 5 +++-- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/builtin/replace.c b

[PATCH v2 5/8] commit-graph: not compatible with replace objects

2018-08-20 Thread Derrick Stolee
-graph. However, we do ensure that we write a new commit-graph by setting read_replace_refs to 0, thereby ignoring the replace refs. Signed-off-by: Derrick Stolee --- builtin/commit-graph.c | 4 commit-graph.c | 21 + replace-object.c| 2 +- replace

[PATCH v2 3/8] commit-graph: update design document

2018-08-20 Thread Derrick Stolee
to and reads from the commit-graph file when these other features exist. Signed-off-by: Derrick Stolee --- Documentation/technical/commit-graph.txt | 18 +++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/Documentation/technical/commit-graph.txt b/Documentation/technical

[PATCH v2 4/8] test-repository: properly init repo

2018-08-20 Thread Derrick Stolee
Signed-off-by: Derrick Stolee --- t/helper/test-repository.c | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/t/helper/test-repository.c b/t/helper/test-repository.c index 2762ca6562..6a84a53efb 100644 --- a/t/helper/test-repository.c +++ b/t/helper/test

[PATCH v2 7/8] commit-graph: not compatible with uninitialized repo

2018-08-20 Thread Derrick Stolee
Signed-off-by: Derrick Stolee --- commit-graph.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/commit-graph.c b/commit-graph.c index c4eaedf4e5..cee2caab5c 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -62,6 +62,9 @@ extern int read_replace_refs; static int

[PATCH v2 6/8] commit-graph: not compatible with grafts

2018-08-20 Thread Derrick Stolee
Augment commit_graph_compatible(r) to return false when the given repository r has commit grafts or is a shallow clone. Test that in these situations we ignore existing commit-graph files and we do not write new commit-graph files. Helped-by: Jakub Narebski Signed-off-by: Derrick Stolee

[PATCH v2 0/8] Clarify commit-graph and grafts/replace/shallow incompatibilities

2018-08-20 Thread Derrick Stolee
...@mail.gmail.com/ Stefan's response recommending we keep the first two commits. Derrick Stolee (6): commit-graph: update design document test-repository: properly init repo commit-graph: not compatible with replace objects commit-graph: not compatible with grafts commit-graph

Re: What's cooking in git.git (Aug 2018, #04; Fri, 17)

2018-08-20 Thread Derrick Stolee
On 8/20/2018 1:26 PM, Junio C Hamano wrote: Jonathan Nieder writes: Usually, I refrain from merging larger topics in 'next' down to 'master' when we get close to -rc0, but I am wondering if it is better to merge all of them to 'master', even the ones on the larger and possibly undercooked

<    1   2   3   4   5   6   7   8   9   10   >