Re: [PATCH v6 00/14] Serialized Git Commit Graph

On 3/19/2018 8:55 AM, Derrick Stolee wrote: Thanks for this! Fixing this performance problem is very important to me, as we will use the "--stdin-packs" mechanism in the GVFS scenario (we will walk all commits in the prefetch packs full of commits and trees instead of relyi

Re: What's cooking in git.git (Mar 2018, #03; Wed, 14)

On 3/15/2018 4:36 AM, Ævar Arnfjörð Bjarmason wrote: On Thu, Mar 15 2018, Junio C. Hamano jotted: * nd/repack-keep-pack (2018-03-07) 6 commits - SQUASH??? - pack-objects: display progress in get_object_details() - pack-objects: show some progress when counting kept objects - gc --auto:

Re: [PATCH v6 07/14] commit-graph: implement 'git-commit-graph write'

On 3/19/2018 10:36 AM, Ævar Arnfjörð Bjarmason wrote: On Mon, Mar 19 2018, Derrick Stolee jotted: On 3/18/2018 9:25 AM, Ævar Arnfjörð Bjarmason wrote: On Wed, Mar 14 2018, Derrick Stolee jotted: +'git commit-graph write' [--object-dir ] + + +DESCRIPTION +--- + +Manage

Re: [PATCH v6 07/14] commit-graph: implement 'git-commit-graph write'

On 3/18/2018 9:25 AM, Ævar Arnfjörð Bjarmason wrote: On Wed, Mar 14 2018, Derrick Stolee jotted: +'git commit-graph write' [--object-dir ] + + +DESCRIPTION +--- + +Manage the serialized commit graph file. + + +OPTIONS +--- +--object-dir:: + Use given directory

Re: [PATCH v6 12/14] commit-graph: read only from specific pack-indexes

On 3/15/2018 6:50 PM, SZEDER Gábor wrote: On Wed, Mar 14, 2018 at 8:27 PM, Derrick Stolee <sto...@gmail.com> wrote: From: Derrick Stolee <dsto...@microsoft.com> Teach git-commit-graph to inspect the objects only in a certain list of pack-indexes within the given pack directory.

Re: [PATCH v6 00/14] Serialized Git Commit Graph

On 3/16/2018 12:28 PM, Lars Schneider wrote: On 14 Mar 2018, at 21:43, Junio C Hamano <gits...@pobox.com> wrote: Derrick Stolee <sto...@gmail.com> writes: Hopefully this version is ready to merge. I have several follow-up topics in mind to submit soon after, including: A few

Re: [PATCH v6 00/14] Serialized Git Commit Graph

On 3/16/2018 4:19 PM, Jeff King wrote: On Fri, Mar 16, 2018 at 04:06:39PM -0400, Jeff King wrote: Furthermore, in order to look at an object it has to be zlib inflated first, and since commit objects tend to be much smaller than trees and especially blobs, there are a lot less bytes to

[PATCH v6 01/14] csum-file: rename hashclose() to finalize_hashfile()

From: Derrick Stolee <dsto...@microsoft.com> The hashclose() method behaves very differently depending on the flags parameter. In particular, the file descriptor is not always closed. Perform a simple rename of "hashclose()" to "finalize_hashfile()" in preparation for f

[PATCH v6 07/14] commit-graph: implement 'git-commit-graph write'

From: Derrick Stolee <dsto...@microsoft.com> Teach git-commit-graph to write graph files. Create new test script to verify this command succeeds without failure. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 39 bu

[PATCH v6 05/14] commit-graph: create git-commit-graph builtin

From: Derrick Stolee <dsto...@microsoft.com> Teach git the 'commit-graph' builtin that will be used for writing and reading packed graph files. The current implementation is mostly empty, except for an '--object-dir' option. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> ---

[PATCH v6 13/14] commit-graph: build graph from starting commits

From: Derrick Stolee <dsto...@microsoft.com> Teach git-commit-graph to read commits from stdin when the --stdin-commits flag is specified. Commits reachable from these commits are added to the graph. This is a much faster way to construct the graph than inspecting all packed o

[PATCH v6 14/14] commit-graph: implement "--additive" option

From: Derrick Stolee <dsto...@microsoft.com> Teach git-commit-graph to add all commits from the existing commit-graph file to the file about to be written. This should be used when adding new commits without performing garbage collection. Signed-off-by: Derrick Stolee <dsto...@micr

[PATCH v6 12/14] commit-graph: read only from specific pack-indexes

From: Derrick Stolee <dsto...@microsoft.com> Teach git-commit-graph to inspect the objects only in a certain list of pack-indexes within the given pack directory. This allows updating the commit graph iteratively. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Docum

[PATCH v6 11/14] commit: integrate commit graph with commit parsing

From: Derrick Stolee <dsto...@microsoft.com> Teach Git to inspect a commit graph file to supply the contents of a struct commit when calling parse_commit_gently(). This implementation satisfies all post-conditions on the struct commit, including loading parents, the root tree, and the commi

[PATCH v6 04/14] graph: add commit graph design document

From: Derrick Stolee <dsto...@microsoft.com> Add Documentation/technical/commit-graph.txt with details of the planned commit graph feature, including future plans. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/technical/commit-gra

[PATCH v6 09/14] commit-graph: add core.commitGraph setting

From: Derrick Stolee <dsto...@microsoft.com> The commit graph feature is controlled by the new core.commitGraph config setting. This defaults to 0, so the feature is opt-in. The intention of core.commitGraph is that a user can always stop checking for or parsing commit graph

[PATCH v6 00/14] Serialized Git Commit Graph

ub-items that appear in priority order. Derrick Stolee (14): csum-file: rename hashclose() to finalize_hashfile() csum-file: refactor finalize_hashfile() method commit-graph: add format document graph: add commit graph design document commit-graph: create git-commit-graph builtin com

[PATCH v6 08/14] commit-graph: implement git commit-graph read

From: Derrick Stolee <dsto...@microsoft.com> Teach git-commit-graph to read commit graph files and summarize their contents. Use the read subcommand to verify the contents of a commit graph file in the tests. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentatio

[PATCH v6 06/14] commit-graph: implement write_commit_graph()

From: Derrick Stolee <dsto...@microsoft.com> Teach Git to write a commit graph file by checking all packed objects to see if they are commits, then store the file in the given object directory. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Makefile | 1 + co

[PATCH v6 10/14] commit-graph: close under reachability

From: Derrick Stolee <dsto...@microsoft.com> Teach write_commit_graph() to walk all parents from the commits discovered in packfiles. This prevents gaps given by loose objects or previously-missed packfiles. Also automatically add commits from the existing graph file, if it exists. Sign

[PATCH v6 02/14] csum-file: refactor finalize_hashfile() method

From: Derrick Stolee <dsto...@microsoft.com> If we want to use a hashfile on the temporary file for a lockfile, then we need finalize_hashfile() to fully write the trailing hash but also keep the file descriptor open. Do this by adding a new CSUM_HASH_IN_STREAM flag along with a func

[PATCH v6 03/14] commit-graph: add format document

From: Derrick Stolee <dsto...@microsoft.com> Add document specifying the binary format for commit graphs. This format allows for: * New versions. * New hash functions and hash lengths. * Optional extensions. Basic header information is followed by a binary table of contents into &

Re: How to debug a "git merge"?

On 3/14/2018 12:56 PM, Lars Schneider wrote: Hi, I am investigating a Git merge (a86dd40fe) in which an older version of a file won over the newer version. I try to understand why this is the case. I can reproduce the merge with the following commands: $ git checkout -b test a02fa3303 $

Re: [PATCH v5 04/13] csum-file: add CSUM_KEEP_OPEN flag

On 3/13/2018 5:42 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: On 2/26/2018 9:32 PM, Derrick Stolee wrote: This patch is new to the series due to the interactions with the lockfile API and the hashfile API. I need to ensure the hashfile writes the hash value at t

Re: [PATCH v5 04/13] csum-file: add CSUM_KEEP_OPEN flag

On 2/26/2018 9:32 PM, Derrick Stolee wrote: This patch is new to the series due to the interactions with the lockfile API and the hashfile API. I need to ensure the hashfile writes the hash value at the end of the file, but keep the file descriptor open so the lock is valid. I welcome any

Re: [PATCH 0/4] Speed up git tag --contains

ion in a faster cache. I had a few proposals over the years, but I won't even bother to dig them up, because there's quite recent and promising work in this area from Derrick Stolee: https://public-inbox.org/git/1519698787-190494-1-git-send-email-dsto...@microsoft.com/ And finally, the thing

Partial Clone: Commands that could be problematic

We discussed partial clone today during the contributor's summit. There were some concerns about some commands that would cause over-hydration of blobs that need server requests to resolve. GVFS blocks "fsck", "gc", "prune", "repack", "submodule", and "worktree" [1]. I promised I would

Re: [RFC] Contributing to Git (on Windows)

I really appreciate the feedback on this document, Jonathan. On 3/3/2018 1:27 PM, Jonathan Nieder wrote: Hi Dscho, Johannes Schindelin wrote: Jonathan Nieder writes: Dereck Stolee wrote: nit: s/Dereck/Derrick/ Is my outgoing email name misspelled, or do you have a

Re: Contributor Summit planning

On 3/3/2018 5:39 AM, Jeff King wrote: On Sat, Mar 03, 2018 at 05:30:10AM -0500, Jeff King wrote: As in past years, I plan to run it like an unconference. Attendees are expected to bring topics for group discussion. Short presentations are also welcome. We'll put the topics on a whiteboard in

Re: What's cooking in git.git (Mar 2018, #01; Thu, 1)

On 3/1/2018 5:20 PM, Junio C Hamano wrote: -- [Graduated to "master"] * jt/binsearch-with-fanout (2018-02-15) 2 commits (merged to 'next' on 2018-02-15 at 7648891022) + packfile: refactor hash search with fanout table + packfile: remove

Re: [RFC] Contributing to Git (on Windows)

On 3/1/2018 11:44 PM, Jonathan Nieder wrote: Hi, Derrick Stolee wrote: Now, we'd like to make that document publicly available. These steps are focused on a Windows user, so we propose putting them in the git-for-windows/git repo under CONTRIBUTING.md. I have a pull request open for feedback

[RFC] Contributing to Git (on Windows)

We (Git devs at Microsoft) have had several people start contributing to Git over the past few years (I'm the most-recent addition). As we on-boarded to Git development on our Windows machines, we collected our setup steps on an internal wiki page. Now, we'd like to make that document

Re: [PATCH v2] sha1_name: fix uninitialized memory errors

On 2/28/2018 3:50 PM, Junio C Hamano wrote: Derrick Stolee <dsto...@microsoft.com> writes: diff --git a/sha1_name.c b/sha1_name.c index 611c7d24dd..a041d8d24f 100644 --- a/sha1_name.c +++ b/sha1_name.c @@ -547,15 +547,15 @@ static void find_abbrev_len_for_pack(struct packed_

Re: [PATCH 00/11] Moving global state into the repository object (part 2)

On 2/27/2018 9:15 PM, Duy Nguyen wrote: On Tue, Feb 27, 2018 at 05:05:57PM -0800, Stefan Beller wrote: This applies on top of origin/sb/object-store and is the continuation of that series, adding the repository as a context argument to functions. This series focusses on packfile handling,

Re: [PATCH 03/11] packfile: allow install_packed_git to handle arbitrary repositories

On 2/27/2018 8:06 PM, Stefan Beller wrote: -void install_packed_git(struct packed_git *pack) +void install_packed_git(struct repository *r, struct packed_git *pack) This is a good thing to do. I'm just making note that this will collide with the new instances of install_packed_git() that I

Re: [PATCH] revision.c: reduce object database queries

On 2/28/2018 1:37 AM, Jeff King wrote: On Tue, Feb 27, 2018 at 03:16:58PM -0800, Junio C Hamano wrote: This code comes originally form 454fbbcde3 (git-rev-list: allow missing objects when the parent is marked UNINTERESTING, 2005-07-10). But later, in aeeae1b771 (revision traversal: allow

[PATCH v2] sha1_name: fix uninitialized memory errors

lt;christian.cou...@gmail.com> Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- sha1_name.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/sha1_name.c b/sha1_name.c index 611c7d24dd..a041d8d24f 100644 --- a/sha1_name.c +++ b/sha1_name.c @@ -547,

[PATCH v5 09/13] commit-graph: close under reachability

Teach write_commit_graph() to walk all parents from the commits discovered in packfiles. This prevents gaps given by loose objects or previously-missed packfiles. Also automatically add commits from the existing graph file, if it exists. Signed-off-by: Derrick Stolee <dsto...@microsoft.

[PATCH v5 08/13] commit-graph: add core.commitGraph setting

The commit graph feature is controlled by the new core.commitGraph config setting. This defaults to 0, so the feature is opt-in. The intention of core.commitGraph is that a user can always stop checking for or parsing commit graph files if core.commitGraph=0. Signed-off-by: Derrick Stolee <d

[PATCH v5 07/13] commit-graph: implement git commit-graph read

Teach git-commit-graph to read commit graph files and summarize their contents. Use the read subcommand to verify the contents of a commit graph file in the tests. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 12 builtin/commit-g

[PATCH v5 02/13] graph: add commit graph design document

Add Documentation/technical/commit-graph.txt with details of the planned commit graph feature, including future plans. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/technical/commit-graph.txt | 164 +++ 1 file changed, 164 inse

[PATCH v5 10/13] commit: integrate commit graph with commit parsing

| 1.35s | 0.32s | -76% | | rev-list --all | 6.7s | 0.83s | -87% | | rev-list --all --objects | 33.0s | 27.5s | -16% | Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- alloc.c | 1 + commit-graph.c

[PATCH v5 00/13] Serialized Git Commit Graph

ting the ODB. You can run your own performance comparisons by toggling the 'core.commitGraph' setting. [1] https://github.com/derrickstolee/git/pull/2 A GitHub pull request containing the latest version of this patch. Derrick Stolee (13): commit-graph: add format document graph: add commit gr

[PATCH v5 05/13] commit-graph: implement write_commit_graph()

Teach Git to write a commit graph file by checking all packed objects to see if they are commits, then store the file in the given object directory. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Makefile | 1 + commit-graph.c

[PATCH v5 11/13] commit-graph: read only from specific pack-indexes

Teach git-commit-graph to inspect the objects only in a certain list of pack-indexes within the given pack directory. This allows updating the commit graph iteratively. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 11 ++- builtin/

[PATCH v5 13/13] commit-graph: implement "--additive" option

Teach git-commit-graph to add all commits from the existing commit-graph file to the file about to be written. This should be used when adding new commits without performing garbage collection. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.tx

[PATCH v5 12/13] commit-graph: build graph from starting commits

, 700,000+ commits were added to the graph file starting from 'master' in 7-9 seconds, depending on the number of packfiles in the repo (1, 24, or 120). Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 14 +- builtin/commit-g

[PATCH v5 03/13] commit-graph: create git-commit-graph builtin

Thanks for the help in getting all the details right in setting up a builtin. -- >8 -- Teach git the 'commit-graph' builtin that will be used for writing and reading packed graph files. The current implementation is mostly empty, except for an '--object-dir' option. Signed-off-by: Derr

[PATCH v5 06/13] commit-graph: implement 'git-commit-graph write'

Teach git-commit-graph to write graph files. Create new test script to verify this command succeeds without failure. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 39 builtin/commit-graph.c | 33 ++ t

[PATCH v5 01/13] commit-graph: add format document

r commit would cause an extra level of indirection for every merge commit. (Octopus merges suffer from this indirection, but they are very rare.) Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/technical/commit-graph-format.txt | 98 + 1 file changed

[PATCH v5 04/13] csum-file: add CSUM_KEEP_OPEN flag

it in the commit that follows. -- >8 -- If we want to use a hashfile on the temporary file for a lockfile, then we need hashclose() to fully write the trailing hash but also keep the file descriptor open. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- csum-file.c | 10 +++

Re: [PATCH v4 03/13] commit-graph: create git-commit-graph builtin

On 2/26/2018 11:25 AM, SZEDER Gábor wrote: Teach git the 'commit-graph' builtin that will be used for writing and reading packed graph files. The current implementation is mostly empty, except for an '--object-dir' option. Since 'git commit-graph' is a builtin command, it shouldn't show up in

[PATCH] sha1_name: fix uninitialized memory errors

. Then nth_packed_object_oid() does not initialize "oid". Use the return value of nth_packed_object_oid() to prevent these errors. Reported-by: Christian Couder <christian.cou...@gmail.com> Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- sha1_name.c | 11 +++ 1 file

Re: [PATCH] commit-graph: fix some "plain integer as NULL pointer" warnings

On 2/24/2018 12:42 AM, René Scharfe wrote: Am 24.02.2018 um 03:24 schrieb Ramsay Jones: diff --git a/commit-graph.c b/commit-graph.c index fc5ee7e99..c2f443436 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -45,7 +45,7 @@ char *get_graph_latest_filename(const char *obj_dir) {

Re: Use of uninitialised value of size 8 in sha1_name.c

On 2/26/2018 5:23 AM, Christian Couder wrote: On Mon, Feb 26, 2018 at 10:53 AM, Jeff King wrote: On Mon, Feb 26, 2018 at 10:04:22AM +0100, Christian Couder wrote: ==21455== Use of uninitialised value of size 8 ==21455==at 0x2D2A73: get_hex_char_from_oid (sha1_name.c:492)

Re: [PATCH] revision.c: reduce object database queries

On 2/24/2018 8:34 PM, Derrick Stolee wrote: In mark_parents_uninteresting(), we check for the existence of an object file to see if we should treat a commit as parsed. The result is to set the "parsed" bit on the commit. Modify the condition to only check has_object_file() if the re

[PATCH] revision.c: reduce object database queries

~60,000 commits, we find the performance of "git status" went from 1.42 seconds to 1.32 seconds, for a relative difference of -7.0%. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- revision.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git

Re: [PATCH v4 04/13] commit-graph: implement write_commit_graph()

On 2/23/2018 2:30 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: jt/binsearch-with-fanout introduces one when there is a 256-entry fanout table (not the case here). The bsearch() method in search.h (and used in pack-write.c:need_large_offset) does not

Re: [PATCH v4 08/13] commit-graph: implement --delete-expired

On 2/23/2018 2:33 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: The (unlikely, but possible) race condition involves two processes (P1 and P2): 1. P1 reads from graph-latest to see commit graph file F1. 2. P2 updates graph-latest to point to F2 and deletes F1.

Re: [PATCH v4 12/13] commit-graph: read only from specific pack-indexes

On 2/21/2018 5:25 PM, Stefan Beller wrote: On Mon, Feb 19, 2018 at 10:53 AM, Derrick Stolee <sto...@gmail.com> wrote: Teach git-commit-graph to inspect the objects only in a certain list of pack-indexes within the given pack directory. This allows updating the commit graph iteratively, si

Re: [PATCH v4 08/13] commit-graph: implement --delete-expired

On 2/22/2018 1:48 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: Teach git-commit-graph to delete the .graph files that are siblings of a newly-written graph file, except for the file referenced by 'graph-latest' at the beginning of the process and the newly-writte

Re: [PATCH v4 07/13] commit-graph: implement --set-latest

On 2/22/2018 1:31 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: static struct opts_commit_graph { const char *obj_dir; const char *graph_file; + int set_latest; } opts; ... @@ -89,6 +106,8 @@ static int graph_write(int argc, const char

Re: [PATCH v4 08/13] commit-graph: implement --delete-expired

On 2/21/2018 4:34 PM, Stefan Beller wrote: On Mon, Feb 19, 2018 at 10:53 AM, Derrick Stolee <sto...@gmail.com> wrote: graph_name = write_commit_graph(opts.obj_dir); if (graph_name) { if (opts.set_latest) set_latest_file(opts.o

Re: [PATCH v4 04/13] commit-graph: implement write_commit_graph()

On 2/20/2018 5:57 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: +#define GRAPH_OID_VERSION_SHA1 1 +#define GRAPH_OID_LEN_SHA1 20 This hardcoded 20 on the right hand side of this #define is probably problematic. Unless you are planning to possibly store truncate

Re: [PATCH v4 03/13] commit-graph: create git-commit-graph builtin

On 2/21/2018 1:58 PM, Junio C Hamano wrote: Junio C Hamano <gits...@pobox.com> writes: Derrick Stolee <sto...@gmail.com> writes: +int cmd_commit_graph(int argc, const char **argv, const char *prefix) +{ + static struct option builtin_commit_g

Re: [PATCH v4 02/13] graph: add commit graph design document

On 2/20/2018 4:42 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: +2. Walking the entire graph to avoid topological order mistakes. You have at least one more mention of "topological order mistakes" below, but we commonly refer to this issue and blame it

Re: Bug: git log: boundary commits do not respect order (e.g. date-order, topo-order) 2

On 2/21/2018 6:57 PM, Josh Tepper wrote: When using git log, boundary commits (ie, those commits added by specifying --boundary) do not respect the order (e.g., --date-order, --topo-order). Consider the following commit history, where number indicates the order of the commit timestamps:

Re: [PATCH] commit: drop uses of get_cached_commit_buffer()

On 2/21/2018 6:13 PM, Jeff King wrote: On Wed, Feb 21, 2018 at 02:17:11PM -0500, Derrick Stolee wrote: The get_cached_commit_buffer() method provides access to the buffer loaded for a struct commit, if it was ever loadead and was not freed. Two places use this to inform how to output

Re: [PATCH] commit: drop uses of get_cached_commit_buffer()

On 2/21/2018 2:17 PM, Derrick Stolee wrote: The get_cached_commit_buffer() method provides access to the buffer loaded for a struct commit, if it was ever loadead and was not freed. Two places use this to inform how to output information about commits. log-tree.c uses this method to short

Re: [PATCH v4 01/13] commit-graph: add format document

On 2/21/2018 2:23 PM, Stefan Beller wrote: On Mon, Feb 19, 2018 at 10:53 AM, Derrick Stolee <sto...@gmail.com> wrote: +In order to allow extensions that add extra data to the graph, we organize +the body into "chunks" and provide a binary lookup table at the beginning +of the

[PATCH] commit: drop uses of get_cached_commit_buffer()

a buffer and hence rev-list will not output the verbose information. These behaviors are undocumented, untested, and unlikely to be expected by users or other software attempting to parse this output. Helped-by: Jeff King <p...@peff.net> Signed-off-by: Derrick Stolee <dsto...@micr

Re: Question about get_cached_commit_buffer()

On 2/20/2018 5:57 PM, Jeff King wrote: On Tue, Feb 20, 2018 at 05:12:50PM -0500, Derrick Stolee wrote: In rev-list, the "--header" option outputs a value and expects the buffer to be cached. It outputs the header info only if get_cached_commit_buffer() returns a non-null buff

Question about get_cached_commit_buffer()

While working on my commit-graph patch [1] and using a local build in my usual workflows, I found a bug in my branch. Essentially, when calling `git rev-list --header`, the header information is actually missing for many commits except the first. This was not caught in my testing since

[PATCH v4 07/13] commit-graph: implement --set-latest

obust than relying on directory scanning and modified times. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 10 ++ builtin/commit-graph.c | 26 -- commit-graph.c | 7 +++

[PATCH v4 04/13] commit-graph: implement write_commit_graph()

Teach Git to write a commit graph file by checking all packed objects to see if they are commits, then store the file in the given object directory. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Makefile | 1 + commit-graph.c

[PATCH v4 05/13] commit-graph: implement 'git-commit-graph write'

Teach git-commit-graph to write graph files. Create new test script to verify this command succeeds without failure. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 40 + builtin/commit-graph.c | 43 +- t

[PATCH v4 02/13] graph: add commit graph design document

Add Documentation/technical/commit-graph.txt with details of the planned commit graph feature, including future plans. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/technical/commit-graph.txt | 185 +++ 1 file changed, 185 inse

[PATCH v4 10/13] commit-graph: close under reachability

Teach write_commit_graph() to walk all parents from the commits discovered in packfiles. This prevents gaps given by loose objects or previously-missed packfiles. Also automatically add commits from the existing graph file, if it exists. Signed-off-by: Derrick Stolee <dsto...@microsoft.

[PATCH v4 12/13] commit-graph: read only from specific pack-indexes

Teach git-commit-graph to inspect the objects only in a certain list of pack-indexes within the given pack directory. This allows updating the commit graph iteratively, since we add all commits stored in a previous commit graph. Signed-off-by: Derrick Stolee <dsto...@microsoft.

[PATCH v4 11/13] commit: integrate commit graph with commit parsing

.7s | -88% | | branch -vv | 0.42s | 0.27s | -35% | | rev-list --all | 6.4s | 1.0s | -84% | | rev-list --all --objects | 32.6s | 27.6s | -15% | Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- alloc.c

[PATCH v4 13/13] commit-graph: build graph from starting commits

commits. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 15 ++- builtin/commit-graph.c | 27 +-- commit-graph.c | 26 -- commit-graph.h

[PATCH v4 08/13] commit-graph: implement --delete-expired

be using that file. In a multi-process environment, we expect the previoius graph file to be used by a concurrent process, so we do not delete it to avoid race conditions. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 11 +-- builtin/

[PATCH v4 06/13] commit-graph: implement git commit-graph read

Teach git-commit-graph to read commit graph files and summarize their contents. Use the read subcommand to verify the contents of a commit graph file in the tests. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 15 + builtin/commit-g

[PATCH v4 03/13] commit-graph: create git-commit-graph builtin

Teach git the 'commit-graph' builtin that will be used for writing and reading packed graph files. The current implementation is mostly empty, except for an '--object-dir' option. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- .gitignore | 1 + Documen

[PATCH v4 09/13] commit-graph: add core.commitGraph setting

The commit graph feature is controlled by the new core.commitGraph config setting. This defaults to 0, so the feature is opt-in. The intention of core.commitGraph is that a user can always stop checking for or parsing commit graph files if core.commitGraph=0. Signed-off-by: Derrick Stolee <d

[PATCH v4 00/13] Serialized Git Commit Graph

y toggling the 'core.commitgraph' setting. [1] https://public-inbox.org/git/d154319e-bb9e-b300-7c37-27b1dcd2a...@jeffhostetler.com/ Re: What's cooking in git.git (Jan 2018, #03; Tue, 23) [2] https://github.com/derrickstolee/git/pull/2 A GitHub pull request containing the latest version

[PATCH v4 01/13] commit-graph: add format document

r commit would cause an extra level of indirection for every merge commit. (Octopus merges suffer from this indirection, but they are very rare.) Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/technical/commit-graph-format.txt | 90 + 1 file changed

Re: [PATCH v3 04/14] commit-graph: implement write_commit_graph()

On 2/15/2018 1:19 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: +struct packed_oid_list { + struct object_id **list; + int nr; + int alloc; +}; What is the typical access pattern for this data structure? If it is pretty much "allocate and

Re: [PATCH v3 00/14] Serialized Git Commit Graph

On 2/14/2018 1:27 PM, Stefan Beller wrote: On Wed, Feb 14, 2018 at 10:15 AM, Derrick Stolee <sto...@gmail.com> wrote: There has been a lot of interesting discussion on this topic. Some of that involves some decently significant changes from v3, so I wanted to summarize my underst

Re: [PATCH v3 00/14] Serialized Git Commit Graph

There has been a lot of interesting discussion on this topic. Some of that involves some decently significant changes from v3, so I wanted to summarize my understanding of the feedback and seek out more feedback from reviewers before rolling v4. If we have consensus on these topics, then I'll

Re: [PATCH v3 11/14] commit: integrate commit graph with commit parsing

On 2/13/2018 7:12 PM, Jonathan Tan wrote: On Thu, 8 Feb 2018 15:37:35 -0500 Derrick Stolee <sto...@gmail.com> wrote: | Command | Before | After | Rel % | |--|||---| | log --oneline --topo-order -1000 |

Re: What's cooking in git.git (Feb 2018, #02; Tue, 13)

On 2/14/2018 12:23 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: There have been a few "What's cooking" emails since I submitted v1 of "Serialized Git Commit Graph" and it has not appeared with a tracking branch. Is this a mistake, or is it

Re: What's cooking in git.git (Feb 2018, #02; Tue, 13)

On 2/13/2018 8:51 PM, Junio C Hamano wrote: Here are the topics that have been cooking. Commits prefixed with '-' are only in 'pu' (proposed updates) while commits prefixed with '+' are in 'next'. The ones marked with '.' do not appear in any of the integration branches, but I am still holding

Re: [PATCH v2 0/2] Refactor hash search with fanout table

't do it in this set but it definitely should be done. Thanks. I'll try to remember to do that change when this lands in master. I'll bring the new prototype into my commit-graph patch. Reviewed-By: Derrick Stolee <dsto...@microsoft.com>

Re: [PATCH v3 07/14] commit-graph: update graph-head during write

On 2/12/2018 3:37 PM, Junio C Hamano wrote: Junio C Hamano <gits...@pobox.com> writes: Derrick Stolee <sto...@gmail.com> writes: It is possible to have multiple commit graph files in a pack directory, but only one is important at a time. Use a 'graph_head' file to point to the im

Re: [PATCH v3 00/35] protocol version 2

I inspected the diff between v2 and v3 and found the changes to be good. Reviewed-By: Derrick Stolee <dsto...@microsoft.com> Thanks, -Stolee

Re: [PATCH v3 14/14] commit-graph: build graph from starting commits

On 2/9/2018 8:02 AM, SZEDER Gábor wrote: On Thu, Feb 8, 2018 at 9:37 PM, Derrick Stolee <sto...@gmail.com> wrote: Teach git-commit-graph to read commits from stdin when the --stdin-commits flag is specified. Commits reachable from these commits are added to the graph. This is a much fast

Re: [PATCH v3 03/14] commit-graph: create git-commit-graph builtin

On 2/8/2018 4:27 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: Teach git the 'commit-graph' builtin that will be used for writing and reading packed graph files. The current implementation is mostly empty, except for a '--pack-dir' option. Why do we want to use

Re: [PATCH v3 01/14] commit-graph: add format document

On 2/8/2018 4:21 PM, Junio C Hamano wrote: Derrick Stolee <sto...@gmail.com> writes: Add document specifying the binary format for commit graphs. This format allows for: * New versions. * New hash functions and hash lengths. It still is unclear, at least to me, why OID and OID

[PATCH v3 09/14] commit-graph: implement --delete-expired

. Otherwise, they may have an open handle on a graph file that will be deleted by the new call. Signed-off-by: Derrick Stolee <dsto...@microsoft.com> --- Documentation/git-commit-graph.txt | 11 -- builtin/commit-graph.c | 73 -- t/t5318-commit-gr

<    7   8   9   10   11   12   13   14   >