Re: [PATCH v3 02/10] pkt-line: add packet_write function

2017-10-10 Thread Jonathan Tan
On Tue, 3 Oct 2017 13:14:59 -0700 Brandon Williams wrote: > +void packet_write(const int fd_out, const char *buf, size_t size) No need for "const" in "const int fd_out", I think. Same comment for the header file. > +{ > + if (packet_write_gently(fd_out, buf, size)) > +

Re: [PATCH v3 01/10] connect: in ref advertisement, shallows are last

2017-10-10 Thread Jonathan Tan
On Tue, 3 Oct 2017 13:14:58 -0700 Brandon Williams wrote: > +static int process_dummy_ref(void) > +{ > + struct object_id oid; > + const char *name; > + > + if (parse_oid_hex(packet_buffer, , )) > + return 0; > + if (*name != ' ') > +

Re: [PATCH v2] oidmap: map with OID as key

2017-10-03 Thread Jonathan Tan
On Mon, Oct 2, 2017 at 11:31 PM, Jeff King wrote: > Right, I kind of wonder if this has fallen into an uncanny value where > we have this almost-hashmap infrastructure, but the end result is not > significantly easier to use than a plain-old hashmap. > > I.e., it looks like you

Re: [PATCH v6 09/40] Add initial external odb support

2017-10-03 Thread Jonathan Tan
On Tue, Oct 3, 2017 at 2:45 AM, Christian Couder wrote: > Yeah, some people need the faster solution, but my opinion is that > many other people would prefer the single shot protocol. > If all you want to do is a simple resumable clone using bundles for > example, then

Re: [PATCH 00/18] Partial clone (from clone to lazy fetch in 18 patches)

2017-10-03 Thread Jonathan Tan
On Tue, Oct 3, 2017 at 7:39 AM, Jeff Hostetler wrote: > > As I see it there are the following major parts to partial clone: > 1. How to let git-clone (and later git-fetch) specify the desired >subset of objects that it wants? (A ref-relative request.) > 2. How to let

[PATCH v2] oidmap: map with OID as key

2017-09-29 Thread Jonathan Tan
. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- Some replies to v1 [1] [2] seem to indicate that simpler non-duplicated code should be preferred over optimizing away the storage of the 4-byte hash code, and I have no objection to that, so I have updated this code to be a thin w

Re: [PATCH v6 09/40] Add initial external odb support

2017-09-29 Thread Jonathan Tan
On Wed, 27 Sep 2017 18:46:30 +0200 Christian Couder wrote: > I am ok to split the patch series, but I am not sure that 01/40 to > 09/40 is the right range for the first patch series. > I would say that 01/40 to 07/40 is better as it can be seen as a > separate

Re: RFC: Design and code of partial clones (now, missing commits and trees OK)

2017-09-29 Thread Jonathan Tan
On Tue, 26 Sep 2017 17:26:33 +0200 Michael Haggerty wrote: > Maybe naming has been discussed at length before, and I am jumping into > a long-settled topic. And admittedly this is bikeshedding. > > But I find these names obscure, even as a developer. And terms like this >

[PATCH 10/18] pack-objects: rename want_.* to ignore_.*

2017-09-29 Thread Jonathan Tan
ranteed to be in to_pack. [1] For the purposes of pack_objects, a blob is a Git special file if it appears in a to-be-packed tree with a filename beginning with ".git". Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- builtin/pack-objects.c | 56 +

[PATCH 03/18] fsck: support referenced promisor objects

2017-09-29 Thread Jonathan Tan
Teach fsck to not treat missing promisor objects indirectly pointed to by refs as an error when extensions.partialclone is set. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- builtin/fsck.c | 11 +++ t/t0410-partial-clone.sh | 23 +++ 2

[PATCH 06/18] introduce fetch-object: fetch one promisor object

2017-09-29 Thread Jonathan Tan
ow any object type to be "want"-ed. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- Makefile | 1 + builtin/fetch-pack.c | 8 builtin/index-pack.c | 16 +--- fetch-object.c | 23 +++ fetch-object.h | 6 +++

[PATCH 17/18] unpack-trees: batch fetching of missing blobs

2017-09-29 Thread Jonathan Tan
ased on the contents of the local repo alone, whereas for status=delayed, it is the filter process that tells us what needs to be checked in the end. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- fetch-object.c | 27 +++ fetch-object.h | 5 ++

[PATCH 12/18] fetch-pack: support excluding large blobs

2017-09-29 Thread Jonathan Tan
Teach fetch-pack and upload-pack to support excluding large blobs through a blob-max-bytes parameter. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- Documentation/technical/pack-protocol.txt | 9 Documentation/technical/protocol-capabilities.txt | 7 ++ b

[PATCH 18/18] fetch-pack: restore save_commit_buffer after use

2017-09-29 Thread Jonathan Tan
and did not mention parsing the unparsed commit, so I prefer to preserve the existing behavior. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- fetch-pack.c | 4 1 file changed, 4 insertions(+) diff --git a/fetch-pack.c b/fetch-pack.c index 19b8e9322..8e6f54547 100644 --- a

[PATCH 15/18] clone: support excluding large blobs

2017-09-29 Thread Jonathan Tan
Teach clone to support excluding large blobs through a blob-max-bytes parameter. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- builtin/clone.c | 23 +-- t/t5601-clone.sh | 49 + 2 files changed, 70 insertions

[PATCH 11/18] pack-objects: support --blob-max-bytes

2017-09-29 Thread Jonathan Tan
exclusion occurs.) If bitmaps are to be used, we would not know if a blob corresponded to a file whose name starts with ".git". For this reason, when invoked with --blob-max-bytes, pack-objects will not use bitmaps. Signed-off-by: Jonathan Tan <jonathanta...@google.com>

[PATCH 07/18] sha1_file: support lazily fetching missing objects

2017-09-29 Thread Jonathan Tan
) - builtin/prune-packed - check if object to be pruned is packed (if not, don't prune it) - revision - used to exclude packed objects if requested by user - diff - just for optimization Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- builtin/cat-file.c | 2 ++ builtin

[PATCH 09/18] gc: do not repack promisor packfiles

2017-09-29 Thread Jonathan Tan
Teach gc to stop traversal at promisor objects, and to leave promisor packfiles alone. This has the effect of only repacking non-promisor packfiles, and preserves the distinction between promisor packfiles and non-promisor packfiles. Signed-off-by: Jonathan Tan <jonathanta...@google.

[PATCH 14/18] fetch: support excluding large blobs

2017-09-29 Thread Jonathan Tan
Teach fetch to support excluding large blobs through a blob-max-bytes parameter. This is only allowed for the remote configured in extensions.partialclone. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- builtin/fetch.c | 22 -- connected.c

[PATCH 04/18] fsck: support promisor objects as CLI argument

2017-09-29 Thread Jonathan Tan
Teach fsck to not treat missing promisor objects provided on the CLI as an error when extensions.partialclone is set. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- builtin/fsck.c | 2 ++ t/t0410-partial-clone.sh | 13 + 2 files changed, 15 insertions(+)

[PATCH 16/18] clone: configure blobmaxbytes in created repos

2017-09-29 Thread Jonathan Tan
Teach clone to configure blobmaxbytes in any repos that it generates when the --blob-max-bytes parameter is set. Also teach fetch to use this parameter. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- builtin/clone.c | 1 + builtin/fetch.c | 4 re

[PATCH 05/18] index-pack: refactor writing of .keep files

2017-09-29 Thread Jonathan Tan
In a subsequent commit, index-pack will be taught to write ".promisor" files which are similar to the ".keep" files it knows how to write. Refactor the writing of ".keep" files, so that the implementation of writing ".promisor" files becomes easier.

[PATCH 08/18] rev-list: support termination at promisor objects

2017-09-29 Thread Jonathan Tan
in process_blob() and process_tree() do not need to be changed because those happen only when there is a conflict between the expected type and the existing object. If the object doesn't exist, an object will be synthesized, which is fine.) Signed-off-by: Jonathan Tan <jonathanta...@google.

[PATCH 02/18] fsck: support refs pointing to promisor objects

2017-09-29 Thread Jonathan Tan
Teach fsck to not treat refs referring to missing promisor objects as an error when extensions.partialclone is set. For the purposes of warning about no default refs, such refs are still treated as legitimate refs. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- builtin/

[PATCH 13/18] fetch: refactor calculation of remote list

2017-09-29 Thread Jonathan Tan
Separate out the calculation of remotes to be fetched from and the actual fetching. This will allow us to include an additional step before the actual fetching in a subsequent commit. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- builtin/fetch.c | 14 -- 1 file c

[PATCH 01/18] fsck: introduce partialclone extension

2017-09-29 Thread Jonathan Tan
commits, fsck will be taught about other cases. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- Documentation/technical/repository-version.txt | 12 builtin/fsck.c | 2 +- cache.h| 5 +- environ

[PATCH 00/18] Partial clone (from clone to lazy fetch in 18 patches)

2017-09-29 Thread Jonathan Tan
l packfiles downloaded in this way are annotated with ".promisor". Fetching with `git fetch` - The fetch-pack/upload-pack protocol has also been extended to support omission of blobs above a certain size. The client only allows this when fetching from the promiso

Re: [PATCH 07/13] object-filter: common declarations for object filtering

2017-09-29 Thread Jonathan Tan
On Thu, 28 Sep 2017 10:33:39 -0400 Jeff Hostetler wrote: > Maybe. What I have here now is the result of adding these arguments to > rev-list and pack-objects (in the current patch series), and also to > fetch-pack, fetch, clone, upload-pack, index-pack, and the transport

Re: [PATCH] oidmap: map with OID as key

2017-09-29 Thread Jonathan Tan
On Thu, 28 Sep 2017 16:05:57 -0400 Jeff King wrote: > When I saw that you were implementing "oidset" in terms of "oidmap", I > was all ready to be crabby about this extra memory. But then I saw that > the implementation tries hard not to waste any memory. :) > > All of which is

Re: RFC: Design and code of partial clones (now, missing commits and trees OK)

2017-09-28 Thread Jonathan Tan
On Fri, 15 Sep 2017 13:43:43 -0700 Jonathan Tan <jonathanta...@google.com> wrote: > For those interested in partial clones and/or missing objects in repos, > I've updated my original partialclone patches to not require an explicit > list of promises. Fetch/clone still only pe

Re: [PATCH] oidmap: map with OID as key

2017-09-28 Thread Jonathan Tan
On Wed, 27 Sep 2017 17:41:37 -0700 Brandon Williams <bmw...@google.com> wrote: > On 09/27, Jonathan Tan wrote: > > This is similar to using the hashmap in hashmap.c, but with an > > easier-to-use API. In particular, custom entry comparisons no longer > > need to

Re: [PATCH] oidmap: map with OID as key

2017-09-28 Thread Jonathan Tan
On Thu, 28 Sep 2017 12:13:00 +0900 Junio C Hamano <gits...@pobox.com> wrote: > Jonathan Tan <jonathanta...@google.com> writes: > > > This is similar to using the hashmap in hashmap.c, but with an > > easier-to-use API. In particular, custom entry comparisons no

Re: [PATCH 07/13] object-filter: common declarations for object filtering

2017-09-27 Thread Jonathan Tan
On Wed, 27 Sep 2017 13:09:42 -0400 Jeff Hostetler <g...@jeffhostetler.com> wrote: > On 9/26/2017 6:39 PM, Jonathan Tan wrote: > > On Fri, 22 Sep 2017 20:30:11 + > > Jeff Hostetler <g...@jeffhostetler.com> wrote: > > > >> Makefile

[PATCH] oidmap: map with OID as key

2017-09-27 Thread Jonathan Tan
This is similar to using the hashmap in hashmap.c, but with an easier-to-use API. In particular, custom entry comparisons no longer need to be written, and lookups can be done without constructing a temporary entry structure. oidset has been updated to use oidmap. Signed-off-by: Jonathan Tan

Re: [PATCH 03/13] list-objects: filter objects in traverse_commit_list

2017-09-27 Thread Jonathan Tan
On Wed, 27 Sep 2017 15:09:43 -0400 Jeff Hostetler wrote: > By adding it to the set of provisionally omitted objects, we > have the option to capture a little extra information with it > and refer to that the next time we see the object in the traversal. > For example, in

Re: [PATCH 03/13] list-objects: filter objects in traverse_commit_list

2017-09-27 Thread Jonathan Tan
On Wed, 27 Sep 2017 13:04:42 -0400 Jeff Hostetler wrote: > The sparse filter is looking at pathnames and using the same rules > as sparse-checkout to decide which to *include* in the result. This > is essentially backwards from the other filters which are looking for >

Re: [PATCH 09/13] rev-list: add object filtering support

2017-09-26 Thread Jonathan Tan
On Fri, 22 Sep 2017 20:30:13 + Jeff Hostetler wrote: > + if (filter_options.relax) { Add some documentation about how this differs from ignore_missing_links in struct rev_info.

Re: [PATCH 07/13] object-filter: common declarations for object filtering

2017-09-26 Thread Jonathan Tan
On Fri, 22 Sep 2017 20:30:11 + Jeff Hostetler wrote: > Makefile| 1 + > object-filter.c | 269 > > object-filter.h | 173 > 3 files changed, 443 insertions(+) >

Re: [PATCH 03/13] list-objects: filter objects in traverse_commit_list

2017-09-26 Thread Jonathan Tan
On Fri, 22 Sep 2017 20:26:22 + Jeff Hostetler wrote: > From: Jeff Hostetler > > Create traverse_commit_list_filtered() and add filtering You mention _filtered() here, but this patch contains _worker(). > list-objects.h | 30

Re: [PATCH 02/13] oidset2: create oidset subclass with object length and pathname

2017-09-26 Thread Jonathan Tan
On Fri, 22 Sep 2017 20:26:21 + Jeff Hostetler wrote: > From: Jeff Hostetler > > Create subclass of oidset where each entry has a > field to store the length of the object's content > and an optional pathname. > > This will be used in a

[PATCH v5] connect: in ref advertisement, shallows are last

2017-09-26 Thread Jonathan Tan
on other lines. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- Changes in v5: - print warning when encountering capabilities on other lines instead of ignoring them (also updated commit message) - explicitly disallow refs of name "capabilities^{}" (except when it is

Re: RFC: Design and code of partial clones (now, missing commits and trees OK) (part 3)

2017-09-26 Thread Jonathan Tan
On Tue, 26 Sep 2017 10:25:16 -0400 Jeff Hostetler wrote: > >> Perhaps you could augment the OID lookup to remember where the object > >> was found (essentially a .promisor bit set). Then you wouldn't need > >> to touch them all. > > > > Sorry - I don't understand this.

Re: [PATCH 00/13] RFC object filtering for parital clone

2017-09-22 Thread Jonathan Tan
On Fri, 22 Sep 2017 20:26:19 + Jeff Hostetler wrote: > This draft contains filters to: > () omit all blobs > () omit blobs larger than some size > () omit blobs using a sparse-checkout specification > > In addition to specifying the filter criteria, the rev-list

Re: RFC: Design and code of partial clones (now, missing commits and trees OK) (part 3)

2017-09-22 Thread Jonathan Tan
On Fri, 22 Sep 2017 17:32:00 -0400 Jeff Hostetler wrote: > I guess I'm afraid that the first call to is_promised() is going > cause a very long pause as it loads up a very large hash of objects. Yes, the first call will cause a long pause. (I think fsck and gc can

Re: RFC: Design and code of partial clones (now, missing commits and trees OK) (part 2/3)

2017-09-22 Thread Jonathan Tan
On Fri, 22 Sep 2017 17:19:50 -0400 Jeff Hostetler wrote: > > In your specific example, how would rev-list know, on the client, to > > include (or exclude) a large blob in its output if it does not have it, > > and thus does not know its size? > > > > The client doesn't

Re: RFC: Design and code of partial clones (now, missing commits and trees OK)

2017-09-22 Thread Jonathan Tan
On Fri, 22 Sep 2017 17:02:11 -0400 Jeff Hostetler wrote: > > I was struggling a bit with the terminology, true. > > > > Right now I'm thinking of: > > - promisor remote (as you defined) > > - promisor packfile (as you defined) > > - promisor object is an object

Re: [PATCH v4] connect: in ref advertisement, shallows are last

2017-09-22 Thread Jonathan Tan
On Fri, 22 Sep 2017 14:01:04 -0700 Brandon Williams wrote: > > +static void process_capabilities(int len) > > +{ > > + int nul_location = strlen(packet_buffer); > > It may make more sense to not rely on accessing a global buffer here > directly and instead pass in the buff

[PATCH v4] connect: in ref advertisement, shallows are last

2017-09-22 Thread Jonathan Tan
it easier to teach get_remote_heads() to interpret other lines in the ref advertisement, which will be done in a subsequent patch. As part of this change, this patch interprets capabilities only on the first line in the ref advertisement, ignoring all others. Signed-off-by: Jonathan Tan <jonatha

[PATCH v3] connect: in ref advertisement, shallows are last

2017-09-21 Thread Jonathan Tan
it easier to teach get_remote_heads() to interpret other lines in the ref advertisement, which will be done in a subsequent patch. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- I sent the wrong version of this patch :-( This should be the correct one. A bit less clean bec

[PATCH v2] connect: in ref advertisement, shallows are last

2017-09-21 Thread Jonathan Tan
it easier to teach get_remote_heads() to interpret other lines in the ref advertisement, which will be done in a subsequent patch. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- In some in-office discussion, I was informed that my original patch relaxed the ordering of ".keep&q

Re: RFC: Design and code of partial clones (now, missing commits and trees OK) (part 3)

2017-09-21 Thread Jonathan Tan
On Thu, 21 Sep 2017 14:00:40 -0400 Jeff Hostetler wrote: > (part 3) > > Additional overall comments on: > https://github.com/jonathantanmy/git/commits/partialclone2 > > {} WRT the code in is_promised() [1] > > [1] >

Re: RFC: Design and code of partial clones (now, missing commits and trees OK) (part 2/3)

2017-09-21 Thread Jonathan Tan
On Thu, 21 Sep 2017 13:59:43 -0400 Jeff Hostetler wrote: > (part 2) > > Additional overall comments on: > https://github.com/jonathantanmy/git/commits/partialclone2 > > {} I think it would help to split the blob-max-bytes filtering and the > promisor/promised

Re: RFC: Design and code of partial clones (now, missing commits and trees OK)

2017-09-21 Thread Jonathan Tan
On Thu, 21 Sep 2017 13:57:30 -0400 Jeff Hostetler wrote: > There's a lot in this patch series. I'm still studying it, but here > are some notes and questions. I'll start with direct responses to > the RFC here and follow up in a second email with specific questions >

[PATCH] connect: in ref advertisement, shallows are last

2017-09-21 Thread Jonathan Tan
it easier to teach get_remote_heads() to interpret other lines in the ref advertisement, which will be done in a subsequent patch. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- It seems that some people are concerned about looseness in interpreting the ref advertisement, s

Re: [PATCH 3/8] daemon: recognize hidden request arguments

2017-09-20 Thread Jonathan Tan
On Wed, 20 Sep 2017 17:24:43 -0700 Jonathan Tan <jonathanta...@google.com> wrote: > On Wed, 13 Sep 2017 14:54:43 -0700 > Brandon Williams <bmw...@google.com> wrote: > > > A normal request to git-daemon is structured as > > "command path/to/repo\0host=..

Re: [PATCH 3/8] daemon: recognize hidden request arguments

2017-09-20 Thread Jonathan Tan
On Wed, 13 Sep 2017 14:54:43 -0700 Brandon Williams wrote: > A normal request to git-daemon is structured as > "command path/to/repo\0host=..\0" and due to a bug in an old version of > git-daemon 73bb33a94 (daemon: Strictly parse the "extra arg" part of the > command,

[PATCH] fast-export: do not copy from modified file

2017-09-20 Thread Jonathan Tan
ast-export is also fixed in this patch. The existing line "C file6 file7" copies the wrong version of file6, but it has coincidentally worked because file7 was subsequently overridden. Reported-by: Juraj Oršulić <juraj.orsu...@fer.hr> Signed-off-by: Jonathan Tan <jonathant

[PATCH for jk/leak-checkers v2] git-compat-util: make UNLEAK less error-prone

2017-09-19 Thread Jonathan Tan
dify its definitions to not leave these semicolons. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- OK, here is the same patch with an updated commit message. --- git-compat-util.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/git-compat-util.h b/git-com

[PATCH for jk/leak-checkers] git-compat-util: make UNLEAK less error-prone

2017-09-19 Thread Jonathan Tan
refore, modify its definitions to cause a compile-time error if invoked without the trailing semicolon. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- Sure, here is the patch. --- git-compat-util.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/git-com

Re: [PATCH v2 10/10] add UNLEAK annotation for reducing leak false positives

2017-09-19 Thread Jonathan Tan
Thanks - this does look like a good thing to have. Sorry for the late comments. The following comments are assuming that we're going to standardize on UNLEAK(var); (with the semicolon). On Fri, 8 Sep 2017 02:38:41 -0400 Jeff King wrote: > +#ifdef SUPPRESS_ANNOTATED_LEAKS >

Re: [PATCH v6 09/40] Add initial external odb support

2017-09-19 Thread Jonathan Tan
I wonder if it's better to get a change like this (PATCH v6 09/40 and any of the previous patches that this depends on) in and then build on it rather than to review the whole patch set at a time. This would reduce ripple effects (of needing to change later patches in a patch set multiple times

Re: [PATCH v5 25/40] external-odb: add 'get_direct' support

2017-09-15 Thread Jonathan Tan
On Fri, 15 Sep 2017 13:24:50 +0200 Christian Couder wrote: > > There are still some nuances. For example, if an external ODB provides > > both a tree and a blob that the tree references, do we fetch the tree in > > order to call "have" on all its blobs, or do we trust

RFC: Design and code of partial clones (now, missing commits and trees OK)

2017-09-15 Thread Jonathan Tan
For those interested in partial clones and/or missing objects in repos, I've updated my original partialclone patches to not require an explicit list of promises. Fetch/clone still only permits exclusion of blobs, but the infrastructure is there for a local repo to support missing trees and

Re: [PATCH v5 25/40] external-odb: add 'get_direct' support

2017-09-14 Thread Jonathan Tan
On Thu, 14 Sep 2017 10:39:35 +0200 Christian Couder wrote: > From the following email: > > https://public-inbox.org/git/20170804145113.5ceaf...@twelve2.svl.corp.google.com/ > > it looks like his work is fundamentally about changing the rules of > connectivity

[PATCH] Remove inadvertently added outgoing/packfile.h

2017-08-28 Thread Jonathan Tan
This empty file was inadvertently introduced in commit 4f39cd8 ("pack: move pack name-related functions", 2017-08-23). Remove this file. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- outgoing/packfile.h | 0 1 file changed, 0 insertions(+), 0 deletions(-) de

[PATCH] Add t/helper/test-write-cache to .gitignore

2017-08-28 Thread Jonathan Tan
This new binary was introduced in commit 3921a0b ("perf: add test for writing the index", 2017-08-21), but a .gitignore entry was not added for it. Add that entry. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- t/helper/.gitignore | 1 + 1 file changed, 1 insertion(+)

Re: [PATCH v5 35/40] Add Documentation/technical/external-odb.txt

2017-08-25 Thread Jonathan Tan
On Fri, 25 Aug 2017 08:14:08 +0200 Christian Couder wrote: > As Git is used by more and more by people having different needs, I > think it is not realistic to expect that we can optimize its object > storage for all these different needs. So a better strategy is to

[PATCH v2] Doc: clarify that pack-objects makes packs, plural

2017-08-22 Thread Jonathan Tan
Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- Thanks. I've reverted the NAME change and used some of your suggestion for the --max-pack-size documentation. --- Documentation/git-pack-objects.txt | 17 +++-- 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/Docume

[PATCH] Doc: clarify that pack-objects makes packs, plural

2017-08-22 Thread Jonathan Tan
The documentation for pack-objects describes that it creates "a packed archive of objects", which is confusing because it may create multiple packs if --max-pack-size is set. Update the documentation to clarify this. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- It to

Re: [PATCH v3 00/23] Move exported packfile funcs to its own file

2017-08-21 Thread Jonathan Tan
On Sat, 19 Aug 2017 23:40:33 -0700 Junio C Hamano wrote: > Junio C Hamano writes: > > > I have to say that this was a painful topic to integrate. > > > > As you may know, the mk/use-size-t-in-zlib topic is being retracted > > and getting rerolled as a

Re: [PATCH v2 00/25] Move exported packfile funcs to its own file

2017-08-18 Thread Jonathan Tan
On Fri, 11 Aug 2017 15:41:28 -0400 Ben Peart wrote: > Nice to see the pack file functions being refactored out. I looked at > the end result and it looked good to me. Thanks. > Do you have the energy to do a similar refactoring for the remaining > public functions

Re: [RFC PATCH] Updated "imported object" design

2017-08-18 Thread Jonathan Tan
On Fri, 18 Aug 2017 10:18:37 -0400 Ben Peart wrote: > > But if there was a good way to refer to the "anti-projection" in a > > virtualized system (that is, the "real" thing or "object" behind the > > "virtual" thing or "image"), then maybe the "virtualized" language is > >

[PATCH v3 04/23] pack: move open_pack_index(), parse_pack_index()

2017-08-18 Thread Jonathan Tan
alloc_packed_git() in packfile.c is duplicated from sha1_file.c. In a subsequent commit, alloc_packed_git() will be removed from sha1_file.c. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- builtin/count-objects.c | 1 + builtin/fsck.c | 1 + builtin/pack-objects.c

[PATCH v3 08/23] pack: move unuse_pack()

2017-08-18 Thread Jonathan Tan
Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- cache.h | 1 - packfile.c | 9 + packfile.h | 1 + sha1_file.c | 9 - 4 files changed, 10 insertions(+), 10 deletions(-) diff --git a/cache.h b/cache.h index a27018210..0313b0b8d 100644 --- a/cache.h +++ b/c

[PATCH v3 09/23] pack: move add_packed_git()

2017-08-18 Thread Jonathan Tan
Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- cache.h | 1 - connected.c | 1 + packfile.c | 53 + packfile.h | 1 + sha1_file.c | 61 - 5 files chang

[PATCH v3 20/23] pack: move find_pack_entry() and make it global

2017-08-18 Thread Jonathan Tan
This function needs to be global as it is used by sha1_file.c and will be used by packfile.c. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- packfile.c | 53 + packfile.h | 2 ++ sha1_file.

[PATCH v3 10/23] pack: move install_packed_git()

2017-08-18 Thread Jonathan Tan
Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- cache.h | 1 - packfile.c | 11 ++- packfile.h | 2 ++ sha1_file.c | 9 - 4 files changed, 12 insertions(+), 11 deletions(-) diff --git a/cache.h b/cache.h index 3625509f9..c4d8bee52 100644 --- a/cache.h

[PATCH v3 12/23] pack: move unpack_object_header_buffer()

2017-08-18 Thread Jonathan Tan
Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- cache.h | 1 - packfile.c | 25 + packfile.h | 2 ++ sha1_file.c | 25 - 4 files changed, 27 insertions(+), 26 deletions(-) diff --git a/cache.h b/cache.h index 63765d481..75c

[PATCH v3 11/23] pack: move {,re}prepare_packed_git and approximate_object_count

2017-08-18 Thread Jonathan Tan
Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- builtin/gc.c | 1 + bulk-checkin.c | 1 + cache.h| 15 connected.c| 2 +- fetch-pack.c | 1 + http-backend.c | 1 + packfile.c | 217 + pack

[PATCH v3 05/23] pack: move release_pack_memory()

2017-08-18 Thread Jonathan Tan
The function unuse_one_window() needs to be temporarily made global. Its scope will be restored to static in a subsequent commit. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- git-compat-util.h | 2 -- packfile.c

[PATCH v3 18/23] pack: move find_pack_entry_one(), is_pack_valid()

2017-08-18 Thread Jonathan Tan
Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- cache.h | 8 --- packfile.c | 76 - packfile.h | 9 ++-- sha1_file.c | 73 -- 4 files chang

[PATCH v3 03/23] pack: move pack_report()

2017-08-18 Thread Jonathan Tan
Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- cache.h | 2 -- packfile.c | 24 packfile.h | 2 ++ sha1_file.c | 24 4 files changed, 26 insertions(+), 26 deletions(-) diff --git a/cache.h b/cache.h index aa2b4d390..a04

[PATCH v3 19/23] pack: move find_sha1_pack()

2017-08-18 Thread Jonathan Tan
Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- cache.h | 3 --- http-push.c | 1 + http-walker.c | 1 + packfile.c| 13 + packfile.h| 3 +++ sha1_file.c | 13 - 6 files changed, 18 insertions(+), 16 deletions(-) diff --git a/cac

[PATCH v3 21/23] pack: move has_sha1_pack()

2017-08-18 Thread Jonathan Tan
Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- builtin/prune-packed.c | 1 + cache.h| 2 -- diff.c | 1 + packfile.c | 6 ++ packfile.h | 2 ++ revision.c | 1 + sha1_file.c| 6 -- 7 files c

[PATCH v3 14/23] pack: move unpack_object_header()

2017-08-18 Thread Jonathan Tan
Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- cache.h | 1 - packfile.c | 26 ++ packfile.h | 1 + sha1_file.c | 26 -- 4 files changed, 27 insertions(+), 27 deletions(-) diff --git a/cache.h b/cache.h index 87f

[PATCH v3 15/23] pack: move clear_delta_base_cache(), packed_object_info(), unpack_entry()

2017-08-18 Thread Jonathan Tan
-by: Jonathan Tan <jonathanta...@google.com> --- cache.h | 7 - packfile.c | 661 ++ packfile.h | 10 + sha1_file.c | 677 ++-- 4 files changed, 685 insertions(+), 670 del

[PATCH v3 22/23] pack: move has_pack_index()

2017-08-18 Thread Jonathan Tan
Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- cache.h | 2 -- packfile.c | 8 packfile.h | 2 ++ sha1_file.c | 8 4 files changed, 10 insertions(+), 10 deletions(-) diff --git a/cache.h b/cache.h index 286891df4..dcbe37a3f 100644 --- a/cache.h +++ b/c

[PATCH v3 17/23] pack: move check_pack_index_ptr(), nth_packed_object_offset()

2017-08-18 Thread Jonathan Tan
Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- cache.h | 16 packfile.c | 33 + packfile.h | 16 sha1_file.c | 33 - 4 files changed, 49 insertions(+), 49 deletions(-) diff

[PATCH v3 13/23] pack: move get_size_from_delta()

2017-08-18 Thread Jonathan Tan
Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- cache.h | 1 - packfile.c | 40 packfile.h | 1 + sha1_file.c | 39 --- 4 files changed, 41 insertions(+), 40 deletions(-) diff --git a/cac

[PATCH v3 16/23] pack: move nth_packed_object_{sha1,oid}

2017-08-18 Thread Jonathan Tan
Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- cache.h | 14 -- packfile.c | 31 +++ packfile.h | 16 +++- sha1_file.c | 31 --- 4 files changed, 46 insertions(+), 46 deletions(-) diff

[PATCH v3 23/23] pack: move for_each_packed_object()

2017-08-18 Thread Jonathan Tan
Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- builtin/cat-file.c | 1 + cache.h| 7 +-- packfile.c | 40 packfile.h | 11 +++ reachable.c| 1 + sha1_file.c

[PATCH v3 02/23] pack: move static state variables

2017-08-18 Thread Jonathan Tan
sha1_file.c declares some static variables that store packfile-related state. Move them to packfile.c. They are temporarily made global, but subsequent commits will restore their scope back to static. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- packfile.c

[PATCH v3 06/23] pack: move pack-closing functions

2017-08-18 Thread Jonathan Tan
The function close_pack_fd() needs to be temporarily made global. Its scope will be restored to static in a subsequent commit. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- builtin/am.c | 1 + builtin/clone.c| 1 + builtin/fetch.c| 1 + builtin/m

[PATCH v3 07/23] pack: move use_pack()

2017-08-18 Thread Jonathan Tan
The function open_packed_git() needs to be temporarily made global. Its scope will be restored to static in a subsequent commit. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- cache.h | 1 - packfile.c

[PATCH v3 01/23] pack: move pack name-related functions

2017-08-18 Thread Jonathan Tan
a corresponding header packfile.h. In this commit, the pack name-related functions are moved. Subsequent commits will move the other functions. Signed-off-by: Jonathan Tan <jonathanta...@google.com> --- Makefile | 1 + builtin/index-pack.c | 1 + builtin/pack-redundant.c | 1 + c

[PATCH v3 00/23] Move exported packfile funcs to its own file

2017-08-18 Thread Jonathan Tan
stripes The merge base commit can be accessed online [1], if you need it. [1] https://github.com/jonathantanmy/git/commits/packmigrate Jonathan Tan (23): pack: move pack name-related functions pack: move static state variables pack: move pack_report() pack: move open_pack_index(

Re: [PATCH] sub-process: print the cmd when a capability is unsupported

2017-08-17 Thread Jonathan Tan
On Thu, 17 Aug 2017 23:34:33 +0200 Lars Schneider wrote: > > > On 17 Aug 2017, at 23:01, Junio C Hamano wrote: > > > > Christian Couder writes: > > > >> ... but I think we should then emphasize more in our test > >>

Re: [RFC PATCH] Updated "imported object" design

2017-08-17 Thread Jonathan Tan
Thanks for your comments. I'll reply to both your e-mails in this one e-mail. > This illustrates another place we need to resolve the > naming/vocabulary. We should at least be consistent to make it easier > to discuss/explain. We obviously went with "virtual" when building > GVFS but I'm OK

Re: [RFC PATCH 01/10] pack: move pack name-related functions

2017-08-16 Thread Jonathan Tan
On Fri, 11 Aug 2017 14:34:27 -0700 Junio C Hamano <gits...@pobox.com> wrote: > Ben Peart <peart...@gmail.com> writes: > > > On 8/9/2017 1:16 PM, Jonathan Tan wrote: > > > >> Ah, I forgot to mention this in the cover letter. I thought that one > >>

Re: [RFC PATCH] Updated "imported object" design

2017-08-16 Thread Jonathan Tan
On Wed, 16 Aug 2017 13:32:23 -0700 Junio C Hamano <gits...@pobox.com> wrote: > Jonathan Tan <jonathanta...@google.com> writes: > > > Also, let me know if there's a better way to send out these patches for > > review. Some of the code here has been reviewed b

<    1   2   3   4   5   6   7   8   9   10   >