Re: [PATCH 0/5] handle corruption in patch-delta

n > > Your related trailing-garbage fix. I split this into two in order to > better demonstrate the cases this part covers. > > [5/5]: patch-delta: handle truncated copy parameters > > My fix for the cp_size read. > > I hope you don't mind me hacking up your patches a bit. Thanks again for > your original report and patch. Looks good to me (feels like traveling back in time). Reviewed-by: Nicolas Pitre > > -Peff >

Re: [PATCH v4] Refactor recv_sideband()

On Wed, 6 Jul 2016, Junio C Hamano wrote: > Thanks; will apply with a miniscule fix. > > > - >8 > > Subject: sideband.c: small optimization of strbuf usage > > > > Signed-off-by: Nicolas Pitre <n...@fluxnic.net> > > ... > > @@ -97,7 +97,7

Re: [PATCH v4] Refactor recv_sideband()

On Wed, 29 Jun 2016, Junio C Hamano wrote: > Nicolas Pitre <n...@fluxnic.net> writes: > > > To make it clearer, here's a patch on top of pu that fixes all the > > issues I think are remaining. All tests pass now. > > Lukas, can you see what is in 'pu' after I

Re: [PATCH v4] Refactor recv_sideband()

On Tue, 28 Jun 2016, Junio C Hamano wrote: > Junio C Hamano writes: > > > It's just that if you take the latter, then the conditional after > > the loop exits (i.e. the last transmission was an incomplete line) > > cannot be "is outbuf empty?", as your base state is "has

Re: [PATCH v4] Refactor recv_sideband()

On Tue, 28 Jun 2016, Junio C Hamano wrote: > Nicolas Pitre <n...@fluxnic.net> writes: > > >> The basic structure of the code (without the "SQUASH" we discussed) > >> looks like this: > >> > >>strbuf_addf(, "%s", PREF

Re: [PATCH v4] Refactor recv_sideband()

On Tue, 28 Jun 2016, Junio C Hamano wrote: > Nicolas Pitre <n...@fluxnic.net> writes: > > >> There is something else going on. I cannot quite explain why I am > >> getting this failure from t5401-update-hooks.sh, for example: > >> > >> ---

Re: [PATCH v4] Refactor recv_sideband()

On Tue, 28 Jun 2016, Junio C Hamano wrote: > Nicolas Pitre <n...@fluxnic.net> writes: > > > Without this, the error and remaining buffer would be reversed as > > mentioned previously. With this, the order is restored, but a newline > > is added to unterm

Re: [PATCH v4] Refactor recv_sideband()

On Tue, 28 Jun 2016, Junio C Hamano wrote: > Nicolas Pitre <n...@fluxnic.net> writes: > > >> When we exit the loop because we set retval to a non-zero value, > >> should we still drain the outbuf? > > > > I would think so. Anything that the remote se

Re: [PATCH v4] Refactor recv_sideband()

On Tue, 28 Jun 2016, Junio C Hamano wrote: > And then that made me stare at the patch even more. We still write > some error messages to stderr in the updated code (without my crap > SQUASH) inside "while (!retval)" loop: > > while (retval == 0) { > int band, len; >

Re: [PATCH v2] Refactor recv_sideband()

On Mon, 27 Jun 2016, Lukas Fleischer wrote: > On Mon, 27 Jun 2016 at 19:50:13, Junio C Hamano wrote: > > Jeff King writes: > > > > > On Mon, Jun 27, 2016 at 08:54:22AM -0700, Junio C Hamano wrote: > > > > > >> It's just you used xwrite() there that introduced a different issue. >

Re: [PATCH v3] Refactor recv_sideband()

On Thu, 23 Jun 2016, Lukas Fleischer wrote: > On Thu, 23 Jun 2016 at 00:47:39, Nicolas Pitre wrote: > > On Wed, 22 Jun 2016, Nicolas Pitre wrote: > > [...] > > > if (*b) { > > > xwrite(STDE

Re: [PATCH v3] Refactor recv_sideband()

On Wed, 22 Jun 2016, Nicolas Pitre wrote: > On Wed, 22 Jun 2016, Lukas Fleischer wrote: > > > Before this patch, we used character buffer manipulations to split > > messages from the sideband at line breaks and insert "remote: " at the > > beginning o

Re: [PATCH v3] Refactor recv_sideband()

output which is then > printed using a single write() call, such that the atomicity of the > output is preserved. See 9ac13ec (atomic write for sideband remote > messages, 2006-10-11) for details. > > Helped-by: Nicolas Pitre <n...@fluxnic.net> > Signed-off-by: Lukas Fleischer <

Re: [PATCH] Refactor recv_sideband()

On Tue, 14 Jun 2016, Lukas Fleischer wrote: > On Tue, 14 Jun 2016 at 19:55:06, Nicolas Pitre wrote: > > It is not buffered as it writes to stderr. And some C libs do separate > > calls to write() for every string format specifier. So "%s%s%c" may end > > up ca

Re: [PATCH] Refactor recv_sideband()

On Tue, 14 Jun 2016, Lukas Fleischer wrote: > Hi Nicolas, > > On Tue, 14 Jun 2016 at 19:09:15, Nicolas Pitre wrote: > > I just looked again at all the contraptions _I_ wrote (not Junio's) for > > a reason why I went to such extremes in making this code co complicated

Re: [PATCH] Refactor recv_sideband()

On Mon, 13 Jun 2016, Nicolas Pitre wrote: > On Mon, 13 Jun 2016, Lukas Fleischer wrote: > > > Improve the readability of recv_sideband() significantly by replacing > > fragile buffer manipulations with more sophisticated format strings. > > Also, reorganize the overall c

Re: [PATCH] Refactor recv_sideband()

On Tue, 14 Jun 2016, Johannes Schindelin wrote: > Hi Nico, > > On Tue, 14 Jun 2016, Nicolas Pitre wrote: > > > On Tue, 14 Jun 2016, Johannes Schindelin wrote: > > > > > On Mon, 13 Jun 2016, Nicolas Pitre wrote: > > > > >

Re: [PATCH] Refactor recv_sideband()

On Tue, 14 Jun 2016, Johannes Schindelin wrote: > Hi, > > On Mon, 13 Jun 2016, Nicolas Pitre wrote: > > > On Mon, 13 Jun 2016, Lukas Fleischer wrote: > > > > > Improve the readability of recv_sideband() significantly by replacing > > > fragile bu

Re: [PATCH] Refactor recv_sideband()

ace a custom implementation of strpbrk() with a call > to the standard C library function. > > Signed-off-by: Lukas Fleischer <lfleisc...@lfos.de> The previous code was a total abomination, even if I happen to know who wrote it. Acked-by: Nicolas Pitre <n...@fluxnic.net> &

Re: [PATCH v2] reduce progress updates in background

On Tue, 14 Apr 2015, Luke Mewburn wrote: Updated patch where is_foreground_fd() is only called in display() just before the output is to be displayed. Acked-by: Nicolas Pitre n...@fluxnic.net -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord

Re: [PATCH] reduce progress updates in background

On Mon, 13 Apr 2015, Luke Mewburn wrote: Hi, I've noticed that when a long-running git operation that generates progress output is suspended and converted to a background process, the terminal still gets spammed with progress updates (to stderr). Many years ago I fixed a similar issue in

Re: [PATCH] reduce progress updates in background

On Tue, 14 Apr 2015, Luke Mewburn wrote: On Mon, Apr 13, 2015 at 10:11:09AM -0400, Nicolas Pitre wrote: | What if you suspend the task and push it into the background? Would be | nice to inhibit progress display in that case, and resume it if the task | returns to the foreground

Re: Pack v4 again..

On Fri, 13 Feb 2015, Duy Nguyen wrote: After taking 1.5 years vacation from pack v4, I plan to do something about it again. Will post more when I have some patches to discuss. Only one question for now (forgive me if I asked already, it's been quite some time) Yeah. I had to re-study my own

Re: [PATCH v2 2/2] Document some functions defined in object.c

On Fri, 28 Feb 2014, Michael Haggerty wrote: Signed-off-by: Michael Haggerty mhag...@alum.mit.edu Acked-by: Nicolas Pitre n...@fluxnic.net --- object.c | 29 - object.h | 7 +++ 2 files changed, 35 insertions(+), 1 deletion(-) diff --git a/object.c b

Re: [PATCH 3/6] find_pack_entry(): document last_found_pack

-by: Michael Haggerty mhag...@alum.mit.edu REviewed-by: Nicolas Pitre n...@fluxnic.net --- sha1_file.c | 16 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/sha1_file.c b/sha1_file.c index 6e8c05d..0910939 100644 --- a/sha1_file.c +++ b/sha1_file.c @@ -60,6

Re: [PATCH 5/6] Document a bunch of functions defined in sha1_file.c

On Fri, 21 Feb 2014, Michael Haggerty wrote: Signed-off-by: Michael Haggerty mhag...@alum.mit.edu Acked-by: Nicolas Pitre n...@fluxnic.net --- cache.h | 66 ++--- sha1_file.c | 26 +--- 2 files changed, 78

Re: [PATCH 6/6] Document some functions defined in object.c

-by: Nicolas Pitre n...@fluxnic.net +/* + * Insert obj into the hash table hash, which has length size (which + * must be a power of 2). On collisions, simply overflow to the next + * empty bucket. + */ static void insert_obj_hash(struct object *obj, struct object **hash, unsigned int size

Re: pack corruption post-mortem

On Sat, 19 Oct 2013, Shawn Pearce wrote: On Sat, Oct 19, 2013 at 7:41 AM, Nicolas Pitre n...@fluxnic.net wrote: On Sat, 19 Oct 2013, Duy Nguyen wrote: On Wed, Oct 16, 2013 at 3:34 PM, Jeff King p...@peff.net wrote: I was recently presented with a repository with a corrupted packfile

Re: pack corruption post-mortem

On Sun, 20 Oct 2013, Duy Nguyen wrote: On Sat, Oct 19, 2013 at 9:41 PM, Nicolas Pitre n...@fluxnic.net wrote: On Sat, 19 Oct 2013, Duy Nguyen wrote: The SHA1 used in the name of the pack file is actually the SHA1 checksum of the SHA1 table. The path and ident tables are already

Re: pack corruption post-mortem

On Sat, 19 Oct 2013, Duy Nguyen wrote: On Wed, Oct 16, 2013 at 3:34 PM, Jeff King p...@peff.net wrote: I was recently presented with a repository with a corrupted packfile, and was asked if the data was recoverable. This post-mortem describes the steps I took to investigate and fix the

Re: pack-object's try_delta fast path for v2 trees?

On Mon, 14 Oct 2013, Jeff King wrote: On Sat, Oct 12, 2013 at 10:42:17AM +0700, Nguyen Thai Ngoc Duy wrote: Just wondering if this has been considered and dropped before. Currently we use try_delta() for every object including trees. But trees are special. All tree entries must be unique

Re: pack-object's try_delta fast path for v2 trees?

On Mon, 14 Oct 2013, Jeff King wrote: By the way, I'm sorry I haven't looked more carefully at the packv4 patches yet. I am excited about it, but I've just got a long queue of other things (and because it's big and challenging, it's easy to put off). ;-) While I consider the format pretty

Re: [BAD PATCH 0/9] v4-aware tree walker API

On Fri, 11 Oct 2013, Duy Nguyen wrote: On Fri, Oct 11, 2013 at 07:22:59PM +0700, Duy Nguyen wrote: Maybe we could make an exception and allow the tree walker to pass pv4_tree_cache* directly to decode_entries so it does not need to do the first lookup every time..

Re: pack-object's try_delta fast path for v2 trees?

On Sat, 12 Oct 2013, Duy Nguyen wrote: Hi, Just wondering if this has been considered and dropped before. Currently we use try_delta() for every object including trees. But trees are special. All tree entries must be unique and sorted. That helps simplify diff algorithm, as demonstrated by

Re: [BAD PATCH 0/9] v4-aware tree walker API

On Wed, 9 Oct 2013, Nguyễn Thái Ngọc Duy wrote: I know I still have a lot of holes to plug, but this was more interesting because we could see some encouraging numbers. Unfortunately the result is disappointing. Maybe I did it in a stupid way and need to restart with a totally different way.

Re: Local tag killer

On Mon, 30 Sep 2013, Marc Branchaud wrote: Why would there be ambiguity warnings? The fetch command shouldn't issue any warnings, since all the remotes' names get safely tucked away in distinct namespaces. Are we talking about DWIM warnings? Aside from git-describe I don't see why such

Re: Local tag killer

On Mon, 30 Sep 2013, Marc Branchaud wrote: On 13-09-30 11:52 AM, Nicolas Pitre wrote: Consider that I have in my Linux kernel tree: - a remote branch corresponding to Linus' master tree - multiple remote branches corresponding to Linux stable branches - a remote for linux-next

Re: Local tag killer

On Mon, 30 Sep 2013, Marc Branchaud wrote: On 13-09-30 04:08 PM, Nicolas Pitre wrote: Again, in the cases where there is actually a SHA1 conflict between all possible tags that match a tag short-end then listing them and asking the user to be more explicit is the right thing to do

Re: Local tag killer

On Mon, 30 Sep 2013, Marc Branchaud wrote: On 13-09-30 06:44 PM, Nicolas Pitre wrote: On Mon, 30 Sep 2013, Marc Branchaud wrote: On 13-09-30 04:08 PM, Nicolas Pitre wrote: Again, in the cases where there is actually a SHA1 conflict between all possible tags that match a tag short

Re: [PATCH 00/10] pack v4 UI support

On Thu, 26 Sep 2013, Duy Nguyen wrote: On Thu, Sep 26, 2013 at 11:51 AM, Nicolas Pitre n...@fluxnic.net wrote: Multi-base tree support is not part of packv4 capability. Let's see if such support comes before the series is merged to master. If so we can drop that line from protocol

Re: Local tag killer

On Tue, 24 Sep 2013, Jeff King wrote: On Sat, Sep 21, 2013 at 08:42:26AM +0200, Michael Haggerty wrote: I think it would be preferable if --prune would *not* affect tags, and if there were an extra option like --prune-tags that would have to be used explicitly to cause tags to be pruned.

Re: [PATCH 00/17] np/pack-v4 updates

On Sat, 21 Sep 2013, Nguyễn Thái Ngọc Duy wrote: This contains many bug fixes or cleanups. Also you can now run the test suite with v4 by setting GIT_TEST_OPTS=--packv4. The test suite passes now. pack size limit is not officially not supported with v4. index-pack also learns to convert

Re: [PATCH 15/17] t5300: avoid testing ofs-delta with --packv4

On Sat, 21 Sep 2013, Nguyễn Thái Ngọc Duy wrote: Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com --- t/t5300-pack-object.sh | 39 ++- 1 file changed, 22 insertions(+), 17 deletions(-) [...] This, in combination with patch 10/17, is making the test

Re: [PATCH 07/17] fixup! index-pack, pack-objects: allow creating .idx v2 with .pack v4

On Sat, 21 Sep 2013, Nguyễn Thái Ngọc Duy wrote: Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com --- sha1_file.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/sha1_file.c b/sha1_file.c index e98eb8b..ef6ecc8 100644 --- a/sha1_file.c +++ b/sha1_file.c @@

Re: [PATCH 15/17] t5300: avoid testing ofs-delta with --packv4

On Sun, 22 Sep 2013, Duy Nguyen wrote: On Sat, Sep 21, 2013 at 11:46 PM, Nicolas Pitre n...@fluxnic.net wrote: On Sat, 21 Sep 2013, Nguy­n Thái Ng÷c Duy wrote: Signed-off-by: Nguy­n Thái Ng÷c Duy pclo...@gmail.com --- t/t5300-pack-object.sh | 39

Re: [PATCH np/pack-v4] index-pack: tighten object type check based on pack version

On Wed, 18 Sep 2013, Nguyễn Thái Ngọc Duy wrote: In pack version 4, ref-delta technically could be used to compress any objects including commits and trees (both canonical and v4). But it does not make sense to do so. It can only lead to performance degradation. Catch those packers.

Git pack v4: next step, help required

I think the pack v4 format and compatibility code is pretty stable now, and probably as optimal as it can reasonably be. @Junio: might be a good idea to refresh your pu branch again. Now the biggest problem to solve is the actual tree deltification. I don't have the time to spend on this

Re: [PATCH 2/4] pack v4: add v4_size to struct delta_base_cache_entry

On Sun, 15 Sep 2013, Duy Nguyen wrote: On Sat, Sep 14, 2013 at 11:22 AM, Nicolas Pitre n...@fluxnic.net wrote: The cache is currently updated by the caller. The caller may ask for a copy of 2 entries from a base object, but that base object may itself copy those objects from somewhere

Re: [PATCH 2/4] pack v4: add v4_size to struct delta_base_cache_entry

On Thu, 12 Sep 2013, Nguyễn Thái Ngọc Duy wrote: The intention is to store flat v4 trees in delta base cache to avoid repeatedly expanding copy sequences in v4 trees. When the user needs to unpack a v4 tree and the tree is found in the cache, the tree will be converted back to canonical

Re: [PATCH 2/4] pack v4: add v4_size to struct delta_base_cache_entry

On Fri, 13 Sep 2013, Duy Nguyen wrote: On Fri, Sep 13, 2013 at 8:27 PM, Nicolas Pitre n...@fluxnic.net wrote: On Thu, 12 Sep 2013, Nguyễn Thái Ngọc Duy wrote: The intention is to store flat v4 trees in delta base cache to avoid repeatedly expanding copy sequences in v4 trees. When

Re: [PATCH] pack-objects: no crc check when the cached version is used

On Sat, 14 Sep 2013, Duy Nguyen wrote: On Sat, Sep 14, 2013 at 4:26 AM, Thomas Rast tr...@inf.ethz.ch wrote: I tried the perf script below, but at least for the git repo the only thing I can see is noise. --stdout does not set do_check_packed_object_crc, you need to run pack-objects

Re: [PATCH 2/4] pack v4: add v4_size to struct delta_base_cache_entry

On Fri, 13 Sep 2013, Nicolas Pitre wrote: On Fri, 13 Sep 2013, Duy Nguyen wrote: On Fri, Sep 13, 2013 at 8:27 PM, Nicolas Pitre n...@fluxnic.net wrote: However I can see that, as you say, the same base object is repeatedly referenced. This means that we need to parse it over and over

Re: [PATCH v2 15/16] index-pack: use nr_objects_final as sha1_table size

On Tue, 10 Sep 2013, Duy Nguyen wrote: On Mon, Sep 9, 2013 at 10:01 PM, Nicolas Pitre n...@fluxnic.net wrote: However this means that the progress meter will now be wrong and that's terrible ! Users *will* complain that the meter doesn't reach 100% and they'll protest for being denied

Re: [PATCH 00/21] np/pack-v4 updates

On Thu, 12 Sep 2013, Duy Nguyen wrote: On Wed, Sep 11, 2013 at 11:25 PM, Nicolas Pitre n...@fluxnic.net wrote: On Wed, 11 Sep 2013, Duy Nguyen wrote: Nico, if you have time you may want to look into this. The result v4 pack from pack-objects on git.git for me is 35MB (one branch) while

Re: [PATCH] lookup_object: remove hashtable_index() and optimize hash_obj()

On Wed, 11 Sep 2013, Jeff King wrote: On Tue, Sep 10, 2013 at 06:17:12PM -0400, Nicolas Pitre wrote: Also remove the modulus as this is an expansive operation. The size argument is always a power of 2 anyway, so a simple mask operation provides the same result. On a 'git rev-list

Re: [PATCH 19/21] index-pack, pack-objects: allow creating .idx v2 with .pack v4

On Wed, 11 Sep 2013, Nguyễn Thái Ngọc Duy wrote: While .idx v3 is recommended because it's smaller, there is no reason why .idx v2 can't use with .pack v4. Enable it, at least for the test suite as some tests need to this kind of information from show-index and show-index does not support

Re: [PATCH 20/21] show-index: acknowledge that it does not read .idx v3

On Wed, 11 Sep 2013, Nguyễn Thái Ngọc Duy wrote: show-index takes .idx from stdin while v3 requires the .pack. It's used for testing purposes only. Let those test scripts force .idx v2 with index-pack. Since I have a patch adding (partial) index v3 support to show-index in my tree, I've

Re: [PATCH 00/21] np/pack-v4 updates

On Wed, 11 Sep 2013, Duy Nguyen wrote: Nico, if you have time you may want to look into this. The result v4 pack from pack-objects on git.git for me is 35MB (one branch) while packv4-create produces 30MB (v2 is 40MB). I don't know why there is such a big difference in size. I compared. Ident

Re: [PATCH 00/21] np/pack-v4 updates

On Wed, 11 Sep 2013, Nguyễn Thái Ngọc Duy wrote: This contains fixups for some of my patches, some of Nico's, adds v4 support to unpack-objects because the test suite needs it. With these, when force generating pack v4 unconditionally, the remaining failed tests are: [...] @junio: I've

Re: pack v4 trees with a canonical base

On Tue, 10 Sep 2013, Junio C Hamano wrote: Duy Nguyen pclo...@gmail.com writes: On Tue, Sep 10, 2013 at 2:25 AM, Nicolas Pitre n...@fluxnic.net wrote: An eventual optimization to index-pack when completing a pack would be to attempt the encoding of appended tree objects into the packv4

Re: pack v4 trees with a canonical base

On Tue, 10 Sep 2013, Junio C Hamano wrote: Nicolas Pitre n...@fluxnic.net writes: On Tue, 10 Sep 2013, Junio C Hamano wrote: There may be trees in the wild that record 100775 or 100777 in the mode field for executable blobs, which also need to be special cased. All the file mode

Re: [PULL REQUEST] initial pack v4 support

On Tue, 10 Sep 2013, Junio C Hamano wrote: Nicolas Pitre n...@fluxnic.net writes: Junio, would you please pull the following into pu: git://git.linaro.org/people/nico/git This is the pack v4 work to date which is somewhat getting usable. It is time it gets more exposure

[PATCH] lookup_object: remove hashtable_index() and optimize hash_obj()

-list --all --objects' run this decreased the time spent in lookup_object from 27.5% to 24.1%. Signed-off-by: Nicolas Pitre n...@fluxnic.net --- I discovered this patch in my git work tree dating from 2 years ago. diff --git a/object.c b/object.c index d8a4b1f..e2dae22 100644 --- a/object.c +++ b

Re: [PULL REQUEST] initial pack v4 support

On Tue, 10 Sep 2013, Junio C Hamano wrote: packv4-parse.c: allow tree entry copying from a canonical tree object This one needed a small fix-up to make it compile. I do not particularly like reusing that size variable, but it seemed to be dead at that point, so...

Re: [PATCH 08/11] pack-objects: create pack v4 tables

On Mon, 9 Sep 2013, Duy Nguyen wrote: On Sun, Sep 8, 2013 at 10:04 PM, Nguyễn Thái Ngọc Duy pclo...@gmail.com wrote: +static void prepare_sha1_table(void) +{ + unsigned i; + /* +* This table includes SHA-1s that may not be present in the +* pack. One of

Re: [PATCH v2 15/16] index-pack: use nr_objects_final as sha1_table size

On Mon, 9 Sep 2013, Nguyễn Thái Ngọc Duy wrote: nr_objects in the next patch is used to reflect the number of actual objects in the stream, which may be smaller than the number recorded in pack header. This highlights an issue that has been nagging me for a while. We decided to send the

Re: [PATCH v2 15/16] index-pack: use nr_objects_final as sha1_table size

On Mon, 9 Sep 2013, Junio C Hamano wrote: Nicolas Pitre n...@fluxnic.net writes: On Mon, 9 Sep 2013, Nguyễn Thái Ngọc Duy wrote: nr_objects in the next patch is used to reflect the number of actual objects in the stream, which may be smaller than the number recorded in pack header

Re: [PATCH v2 15/16] index-pack: use nr_objects_final as sha1_table size

On Mon, 9 Sep 2013, Junio C Hamano wrote: Nicolas Pitre n...@fluxnic.net writes: ... I was wondering if some kind of prefix to the pack stream could be inserted onto the wire when sending a pack v4. Something like: 'T', 'H', 'I', 'N

pack v4 trees with a canonical base

object to be in the canonical representation. This may happen if the encoder detected an irregularity preventing a loss free encoding to the pack v4 format, or if a thin pack was completed with such tree objects. Signed-off-by: Nicolas Pitre n...@fluxnic.net diff --git a/packv4

packv4: current performance

that with specialized string generation functions. This reduced the total runtime by a wopping 40% on a Core2 processor. Signed-off-by: Nicolas Pitre n...@fluxnic.net diff --git a/packv4-parse.c b/packv4-parse.c index 63bba03..c62c4ae 100644 --- a/packv4-parse.c +++ b/packv4-parse.c

Re: [PATCH v2 15/16] index-pack: use nr_objects_final as sha1_table size

On Mon, 9 Sep 2013, Junio C Hamano wrote: Nicolas Pitre n...@fluxnic.net writes: Do we know the actual number of objects to send during the capability negociation? No, and that is not what I meant. We know upfront after capability negotiation (by seeing a request to give them a thin

[PULL REQUEST] initial pack v4 support

SHA-1 table in v4 pack-objects: support writing pack v4 pack v4: support end-of-pack indicator in index-pack and pack-objects index-pack: use nr_objects_final as sha1_table size index-pack: support completing thin packs v4 Nicolas Pitre (41): pack v4: initial pack

Re: [PATCH 05/11] pack-write.c: add pv4_encode_in_pack_object_header

On Sun, 8 Sep 2013, Nguyễn Thái Ngọc Duy wrote: Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com --- pack-write.c | 29 + pack.h | 1 + 2 files changed, 30 insertions(+) diff --git a/pack-write.c b/pack-write.c index 88e4788..6f11104 100644 ---

Re: [PATCH 03/11] pack v4: move packv4-create.c to libgit.a

On Sun, 8 Sep 2013, Nguyễn Thái Ngọc Duy wrote: git-packv4-create now becomes test-packv4. Code that will not be used by pack-objects.c is moved to test-packv4.c. It may be removed when the code transition to pack-objects completes. Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com ---

Re: [PATCH 05/12] index-pack: parse v4 header and dictionaries

On Sat, 7 Sep 2013, Nguyễn Thái Ngọc Duy wrote: Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com --- [...] @@ -1677,6 +1727,15 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix) free(objects); free(index_name_buf); free(keep_name_buf); +

Re: [PATCH 08/12] index-pack: parse v4 tree format

On Sat, 7 Sep 2013, Nguyễn Thái Ngọc Duy wrote: Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com --- builtin/index-pack.c | 104 +-- 1 file changed, 100 insertions(+), 4 deletions(-) diff --git a/builtin/index-pack.c

Re: [PATCH 12/12] index-pack: resolve v4 one-base trees

On Sat, 7 Sep 2013, Nguyễn Thái Ngọc Duy wrote: This is the most common case for delta trees. In fact it's the only kind that's produced by packv4-create. It fits well in the way index-pack resolves deltas and benefits from threading (the set of objects depending on this base does not overlap

Re: [PATCH v2] Document pack v4 format

On Fri, 6 Sep 2013, Duy Nguyen wrote: On Thu, Sep 5, 2013 at 11:52 PM, Duy Nguyen pclo...@gmail.com wrote: On Thu, Sep 5, 2013 at 12:39 PM, Nicolas Pitre n...@fluxnic.net wrote: Now the pack index v3 probably needs to be improved a little, again to accommodate completion of thin packs

Re: [PATCH v3] Document pack v4 format

On Fri, 6 Sep 2013, Duy Nguyen wrote: On Fri, Sep 6, 2013 at 10:23 AM, Nicolas Pitre n...@fluxnic.net wrote: On Fri, 6 Sep 2013, Nguy­n Thái Ng÷c Duy wrote: Signed-off-by: Nguy­n Thái Ng÷c Duy pclo...@gmail.com --- Should be up to date with Nico's latest implementation and also

Re: [PATCH v3] Document pack v4 format

On Fri, 6 Sep 2013, Duy Nguyen wrote: On Fri, Sep 6, 2013 at 8:25 PM, Nicolas Pitre n...@fluxnic.net wrote: On Fri, 6 Sep 2013, Duy Nguyen wrote: On Fri, Sep 6, 2013 at 10:23 AM, Nicolas Pitre n...@fluxnic.net wrote: On Fri, 6 Sep 2013, Nguy­n Thái Ng÷c Duy wrote: Signed-off

Re: [PATCH 10/38] pack v4: commit object encoding

On Thu, 5 Sep 2013, Junio C Hamano wrote: Nicolas Pitre n...@fluxnic.net writes: This goes as follows: - Tree reference: either variable length encoding of the index into the SHA1 table or the literal SHA1 prefixed by 0 (see encode_sha1ref()). - Parent count: variable length

Re: [PATCH 10/38] pack v4: commit object encoding

On Fri, 6 Sep 2013, Junio C Hamano wrote: Nicolas Pitre n...@fluxnic.net writes: OK. If I understand correctly, the committer time stamp is more important than the author's, right? Yeah, it matters a lot more when doing timestamp based traversal without the reachability bitmaps

Re: [PATCH v3] Document pack v4 format

are used for the pack v4 optimized commit and tree representations. I've therefore added this patch to my tree (with the needed changes to the documentation patch): commit 98d4b75aff266015b5dff0a324a2984c2a8f7fa2 Author: Nicolas Pitre n...@fluxnic.net Date: Fri Sep 6 23:45:49 2013 -0400

Re: [PATCH v3] Document pack v4 format

On Fri, 6 Sep 2013, Nicolas Pitre wrote: On Fri, 6 Sep 2013, Duy Nguyen wrote: In your code you reject sha1ref starting with zero too (sha1_file.c::get_base_delta and packv4-parse.c::decode_entries) Yeah... because I wrote the minimum code to make it work with the current encoder

[PATCH 00/38] pack version 4 basic functionalities

After the initial posting here: http://news.gmane.org/group/gmane.comp.version-control.git/thread=233061 This is a repost plus the basic read side working, at least to validate the write side and the pack format itself. And many many bug fixes. This can also be fetched here:

[PATCH 34/38] pack v4: code to retrieve a path component

Because the path dictionary table is located right after the name dictionary table, we currently need to load the later to find the former. Signed-off-by: Nicolas Pitre n...@fluxnic.net --- cache.h| 2 ++ packv4-parse.c | 36 2 files changed, 38

[PATCH 28/38] pack v4: code to load and prepare a pack dictionary table for use

Signed-off-by: Nicolas Pitre n...@fluxnic.net --- packv4-parse.c | 77 ++ 1 file changed, 77 insertions(+) diff --git a/packv4-parse.c b/packv4-parse.c index 299fc48..26894bc 100644 --- a/packv4-parse.c +++ b/packv4-parse.c @@ -28,3 +28,80

[PATCH 32/38] pack v4: parse delta base reference

There is only one type of delta with pack v4. The base reference encoding already handles either an offset (via the pack index) or a literal SHA1. We assume in the literal SHA1 case that the object lives in the same pack, just like with previous pack versions. Signed-off-by: Nicolas Pitre n

[PATCH 33/38] pack v4: we can read commit objects now

Signed-off-by: Nicolas Pitre n...@fluxnic.net --- Makefile | 1 + packv4-parse.c | 1 + packv4-parse.h | 7 +++ sha1_file.c| 10 ++ 4 files changed, 19 insertions(+) create mode 100644 packv4-parse.h diff --git a/Makefile b/Makefile index ba6cafc..22fc276 100644

[PATCH 38/38] packv4-create: add a command line argument to limit tree copy sequences

deltas entirely. This allows for experiments with the delta width and see the influence on pack size vs runtime access cost. Signed-off-by: Nicolas Pitre n...@fluxnic.net --- packv4-create.c | 27 --- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/packv4

[PATCH 35/38] pack v4: decode tree objects

objects without going through the SHA1 search. Signed-off-by: Nicolas Pitre n...@fluxnic.net --- packv4-parse.c | 137 ++--- 1 file changed, 131 insertions(+), 6 deletions(-) diff --git a/packv4-parse.c b/packv4-parse.c index b80b73e..04eab46

[PATCH 31/38] sha1_file.c: make use of decode_varint()

... replacing the equivalent open coded loop. Signed-off-by: Nicolas Pitre n...@fluxnic.net --- sha1_file.c | 14 +++--- 1 file changed, 3 insertions(+), 11 deletions(-) diff --git a/sha1_file.c b/sha1_file.c index a298933..67eb903 100644 --- a/sha1_file.c +++ b/sha1_file.c @@ -1687,20

[PATCH 29/38] pack v4: code to retrieve a name

The name dictionary is loaded if not already done. We know it is located right after the SHA1 table (20 bytes per object) which is itself right after the 12-byte header. Then the index is parsed from the input buffer and a pointer to the corresponding entry is returned. Signed-off-by: Nicolas

[PATCH 37/38] pack v4: introduce escape hatches in the name and path indexes

-by: Nicolas Pitre n...@fluxnic.net --- packv4-create.c | 6 +++--- packv4-parse.c | 28 ++-- 2 files changed, 25 insertions(+), 9 deletions(-) diff --git a/packv4-create.c b/packv4-create.c index fd16222..9d6ffc0 100644 --- a/packv4-create.c +++ b/packv4-create.c

[PATCH 36/38] pack v4: get tree objects

Signed-off-by: Nicolas Pitre n...@fluxnic.net --- packv4-parse.c | 25 + packv4-parse.h | 2 ++ sha1_file.c| 2 +- 3 files changed, 28 insertions(+), 1 deletion(-) diff --git a/packv4-parse.c b/packv4-parse.c index 04eab46..4c218d2 100644 --- a/packv4-parse.c +++ b

[PATCH 10/38] pack v4: commit object encoding

into the pack index file requiring no SHA1 search into the pack index file. Signed-off-by: Nicolas Pitre n...@fluxnic.net --- packv4-create.c | 119 1 file changed, 119 insertions(+) diff --git a/packv4-create.c b/packv4-create.c index 12527c0..d4a79f4

[PATCH 01/38] pack v4: initial pack dictionary structure and code

Signed-off-by: Nicolas Pitre n...@fluxnic.net --- packv4-create.c | 137 1 file changed, 137 insertions(+) create mode 100644 packv4-create.c diff --git a/packv4-create.c b/packv4-create.c new file mode 100644 index 000..2de6d41

[PATCH 22/38] pack index v3

This is a minor change over pack index v2. Since pack v4 already contains the sorted SHA1 table, it is therefore ommitted from the index file. Signed-off-by: Nicolas Pitre n...@fluxnic.net --- pack-write.c| 6 +- packv4-create.c | 10 +- 2 files changed, 14 insertions(+), 2

[PATCH 20/38] pack v4: honor pack.compression config option

Signed-off-by: Nicolas Pitre n...@fluxnic.net --- packv4-create.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/packv4-create.c b/packv4-create.c index c8d3053..45f8427 100644 --- a/packv4-create.c +++ b/packv4-create.c @@ -16,6 +16,7 @@ #include varint.h +static

[PATCH 21/38] pack v4: relax commit parsing a bit

At least commit af25e94d4dcfb9608846242fabdd4e6014e5c9f0 in the Linux kernel repository has author 1120285620 -0700 Signed-off-by: Nicolas Pitre n...@fluxnic.net --- packv4-create.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/packv4-create.c b/packv4-create.c

  1   2   >