Re: [PATCH] t5310-pack-bitmaps: fix bogus 'pack-objects to file can use bitmap' test

2018-08-28 Thread Kirill Smelkov
On Mon, Aug 27, 2018 at 07:04:52PM -0400, Jeff King wrote: > On Mon, Aug 27, 2018 at 10:22:46AM +0000, Kirill Smelkov wrote: > > > A minor comment from outside observer: running tests under something > > like > > > > -e and -o pipefail > > >

Re: [PATCH] t5310-pack-bitmaps: fix bogus 'pack-objects to file can use bitmap' test

2018-08-27 Thread Kirill Smelkov
On Tue, Aug 14, 2018 at 01:47:21PM +0200, SZEDER Gábor wrote: > The test 'pack-objects to file can use bitmap' added in 645c432d61 > (pack-objects: use reachability bitmap index when generating > non-stdout pack, 2016-09-10) is silently buggy and doesn't check what > it's supposed to. > > In

Re: [PATCH] t5500: prettify non-commit tag tests

2018-07-04 Thread Kirill Smelkov
On Tue, Jul 03, 2018 at 12:55:19PM -0400, Jeff King wrote: > I had prepared this as a squash-in for what became c12c9df527, but since > that's now in master, it can go on top (or get dropped, but I think it > is worth it as a style fixup). I'm ok with the patch. I thought it was already squashed

Re: [PATCH] fetch-pack: test explicitly that --all can fetch tag references pointing to non-commits

2018-06-14 Thread Kirill Smelkov
On Thu, Jun 14, 2018 at 09:07:26AM -0700, Junio C Hamano wrote: > Kirill Smelkov writes: > > > Jeff, thanks for corrections. I originally tried to look into invoking > > "git tag" two times, but since git tag always creates a reference it > > would not be s

Re: [PATCH v2] fetch-pack: don't try to fetch peel values with --all

2018-06-13 Thread Kirill Smelkov
On Wed, Jun 13, 2018 at 05:13:02PM -0400, Jeff King wrote: > On Tue, Jun 12, 2018 at 06:54:17PM +0000, Kirill Smelkov wrote: > > > > If an extra connection isn't a problem, you might be better off with > > > "git ls-remote", and then picking through the

Re: [PATCH] fetch-pack: test explicitly that --all can fetch tag references pointing to non-commits

2018-06-13 Thread Kirill Smelkov
On Wed, Jun 13, 2018 at 07:11:47PM -0400, Jeff King wrote: > On Wed, Jun 13, 2018 at 05:05:09PM -0400, Jeff King wrote: > > > > In order to be sure fetching funky tags will never break, let's > > > explicitly test all relevant cases with 4 tag objects pointing to 1) a > > > blob, 2) a tree, 3) a

Re: [PATCH] fetch-pack: test explicitly that --all can fetch tag references pointing to non-commits

2018-06-13 Thread Kirill Smelkov
On Wed, Jun 13, 2018 at 10:42:33AM -0700, Junio C Hamano wrote: > Kirill Smelkov writes: > > > Fetch-pack --all became broken with respect to unusual tags in > > 5f0fc64513 (fetch-pack: eliminate spurious error messages, 2012-09-09), > > and was fixed only recently i

Re: [PATCH] fetch-pack: demonstrate --all failure when remote is empty

2018-06-13 Thread Kirill Smelkov
On Wed, Jun 13, 2018 at 10:13:07AM -0700, Junio C Hamano wrote: > Kirill Smelkov writes: > > > ( Junio, please pick up the patch provided in the end ) > > > > On Tue, Jun 12, 2018 at 06:54:17PM +, Kirill Smelkov wrote: > >> On Tue, Jun 12, 2018 at

[PATCH] fetch-pack: demonstrate --all failure when remote is empty

2018-06-13 Thread Kirill Smelkov
( Junio, please pick up the patch provided in the end ) On Tue, Jun 12, 2018 at 06:54:17PM +, Kirill Smelkov wrote: > On Tue, Jun 12, 2018 at 05:48:49AM -0400, Jeff King wrote: > > On Mon, Jun 11, 2018 at 09:43:02AM +0000, Kirill Smelkov wrote: [...] > > > I'm not sur

[PATCH] fetch-pack: test explicitly that --all can fetch tag references pointing to non-commits

2018-06-13 Thread Kirill Smelkov
gs/tag-to-tree^{} .../git/t/trash directory.t5500-fetch-pack/fetchall$ git fetch-pack --all .. fatal: A git upload-pack: not our ref 038f48ad0beaffbea71d186a05084b79e3870cbf fatal: The remote end hung up unexpectedly Signed-off-by: Kirill Smelkov --- t/t5500-fetch-pack.s

Re: [PATCH v2] fetch-pack: don't try to fetch peel values with --all

2018-06-12 Thread Kirill Smelkov
On Tue, Jun 12, 2018 at 05:48:49AM -0400, Jeff King wrote: > On Mon, Jun 11, 2018 at 09:43:02AM +0000, Kirill Smelkov wrote: > > > > Looking deeper, we do not need these trees and blobs at all. The problem > > > is really just a tag that peels to an object that is not o

Re: [PATCH v2] fetch-pack: don't try to fetch peel values with --all

2018-06-11 Thread Kirill Smelkov
t; [...] > > > Original report and test from Kirill Smelkov. > > > > > > Signed-off-by: Kirill Smelkov > > > Signed-off-by: Jeff King > > > --- > > > diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh > > > @@ -506,30 +506,45

[PATCH] fetch-pack: demonstrate --all breakage when remote have tags to non-commit objects

2018-06-10 Thread Kirill Smelkov
ag] TAGB1 -> TAGB1 * [new tag] TAGB2 -> TAGB2 * [new tag] tag-to-blob -> tag-to-blob # <-- NOTE * [new tag] tag-to-tree -> tag-to-tree # <-- NOTE Signed-o

Re: [PATCH 1/2 v8] pack-objects: respect --local/--honor-pack-keep/--incremental when bitmap is in use

2016-09-13 Thread Kirill Smelkov
On Mon, Sep 12, 2016 at 11:23:18PM -0700, Junio C Hamano wrote: > Kirill Smelkov <k...@nexedi.com> writes: > > > +static int want_found_object(int exclude, struct packed_git *p) > > +{ > > + if (exclude) > > + return 1; > > +

Re: [PATCH] t/perf/run: Don't forget to copy config.mak.autogen & friends to build area

2016-09-13 Thread Kirill Smelkov
exists _and_ is regular file. Some people might have config.mak as a symlink for example. We don't want to miss them too. Please find updated patch below: 8< From: Kirill Smelkov <k...@nexedi.com> Subject: [PATCH] t/perf/run: Don't forget to copy config.mak.autogen & f

[PATCH] t/perf/run: Don't forget to copy config.mak.autogen & friends to build area

2016-09-10 Thread Kirill Smelkov
...@sigill.intra.peff.net/ for example. NOTE config.status has to be copied because otherwise without it the build would want to run reconfigure this way loosing just copied config.mak.autogen. Signed-off-by: Kirill Smelkov <k...@nexedi.com> --- ( Resending as separate patch-mail, just in case ) t/perf/r

[PATCH 2/2 v8] pack-objects: use reachability bitmap index when generating non-stdout pack

2016-09-10 Thread Kirill Smelkov
6(6.58+0.30) 2.87(2.74+0.20) -58.2% More context: http://marc.info/?t=14679210141=1=2 http://public-inbox.org/git/20160707190917.20011-1-k...@nexedi.com/T/#t Cc: Vicent Marti <tan...@gmail.com> Helped-by: Jeff King <p...@peff.net> Signed-off-by: Kirill Smel

[PATCH 1/2 v8] pack-objects: respect --local/--honor-pack-keep/--incremental when bitmap is in use

2016-09-10 Thread Kirill Smelkov
into two parts: finding the object and seeing if it impacts our choice to include it in the pack. We may call the inexpensive want_found_object() twice, but we will never call find_pack_entry_one() if we do not need to. I appreciate help and discussing this change with Junio C Hamano and Jeff King

Re: [PATCH 2/2 v7] pack-objects: use reachability bitmap index when generating non-stdout pack

2016-09-10 Thread Kirill Smelkov
On Thu, Aug 18, 2016 at 02:06:15PM -0400, Jeff King wrote: > On Tue, Aug 09, 2016 at 10:32:17PM +0300, Kirill Smelkov wrote: > > > Subject: Re: [PATCH 2/2 v7] pack-objects: use reachability bitmap index when > >generating non-stdout pack > > This is v7, but as I u

Re: [PATCH 1/2 v5] pack-objects: respect --local/--honor-pack-keep/--incremental when bitmap is in use

2016-09-10 Thread Kirill Smelkov
On Thu, Aug 18, 2016 at 01:52:22PM -0400, Jeff King wrote: > On Tue, Aug 09, 2016 at 10:31:43PM +0300, Kirill Smelkov wrote: > > > Since 6b8fda2d (pack-objects: use bitmaps when packing objects) there > > are two codepaths in pack-objects: with & without using bitmap

[PATCH 2/2 v7] pack-objects: use reachability bitmap index when generating non-stdout pack

2016-08-09 Thread Kirill Smelkov
o teach `git pack-objects file.pack` to use bitmaps instead of using `git pack-objects --stdout >file.pack` + `git index-pack file.pack`. More context: http://marc.info/?t=146792101400001=1=2 Cc: Vicent Marti <tan...@gmail.com> Helped-by: Jeff King <p...@peff.net> Signed-off-by: Ki

[PATCH 1/2 v5] pack-objects: respect --local/--honor-pack-keep/--incremental when bitmap is in use

2016-08-09 Thread Kirill Smelkov
Junio C Hamano and Jeff King. Signed-off-by: Kirill Smelkov <k...@nexedi.com> --- builtin/pack-objects.c | 93 +++-- t/t5310-pack-bitmaps.sh | 92 2 files changed, 152 insertions(+), 33 deletions

Re: [PATCH 1/2] pack-objects: Teach --use-bitmap-index codepath to respect --local, --honor-pack-keep and --incremental

2016-08-09 Thread Kirill Smelkov
On Tue, Aug 09, 2016 at 09:52:18AM -0700, Junio C Hamano wrote: > Kirill Smelkov <k...@nexedi.com> writes: > > > Would you please explain why we should not use touch if we do not care > > about timestamps? Simply style? > > To help readers. > > "

[PATCH 2/2 v6] pack-objects: use reachability bitmap index when generating non-stdout pack

2016-08-09 Thread Kirill Smelkov
o teach `git pack-objects file.pack` to use bitmaps instead of using `git pack-objects --stdout >file.pack` + `git index-pack file.pack`. More context: http://marc.info/?t=146792101400001=1=2 Cc: Vicent Marti <tan...@gmail.com> Helped-by: Jeff King <p...@peff.net> Signed-off-by: Ki

[PATCH 1/2 v4] pack-objects: respect --local/--honor-pack-keep/--incremental when bitmap is in use

2016-08-09 Thread Kirill Smelkov
Junio C Hamano and Jeff King. Signed-off-by: Kirill Smelkov <k...@nexedi.com> --- builtin/pack-objects.c | 93 +++-- t/t5310-pack-bitmaps.sh | 92 2 files changed, 152 insertions(+), 33 deletions

Re: [PATCH v5] pack-objects: teach it to use reachability bitmap index when generating non-stdout pack too

2016-08-09 Thread Kirill Smelkov
On Mon, Aug 08, 2016 at 01:53:20PM -0700, Junio C Hamano wrote: > Kirill Smelkov <k...@nexedi.com> writes: > > > diff --git a/Documentation/config.txt b/Documentation/config.txt > > index bc1c433..4ba0c4a 100644 > > --- a/Documentation/config.txt > > +++ b/Doc

Re: [PATCH 1/2] pack-objects: Teach --use-bitmap-index codepath to respect --local, --honor-pack-keep and --incremental

2016-08-09 Thread Kirill Smelkov
Junio, first of all thanks for feedback, On Mon, Aug 08, 2016 at 12:26:33PM -0700, Junio C Hamano wrote: > Kirill Smelkov <k...@nexedi.com> writes: [...] > > diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c > > index c4c2a3c..e06c1bf 100644 > > --- a/built

Re: [PATCH v4 2/2] pack-objects: Teach it to use reachability bitmap index when generating non-stdout pack too

2016-08-08 Thread Kirill Smelkov
On Mon, Aug 08, 2016 at 11:28:02AM -0700, Junio C Hamano wrote: > Kirill Smelkov <k...@nexedi.com> writes: > > > Another question: I'm preparing another version of "pack-objects: Teach > > --use-bitmap-index codepath to respect --local ..." and was going to &

[PATCH v3] pack-objects: Teach --use-bitmap-index codepath to respect --local, --honor-pack-keep and --incremental

2016-08-08 Thread Kirill Smelkov
Junio C Hamano and Jeff King. Signed-off-by: Kirill Smelkov <k...@nexedi.com> --- builtin/pack-objects.c | 94 ++-- t/t5310-pack-bitmaps.sh | 111 2 files changed, 172 insertions(+), 33 deletions(-) diff

[PATCH v5] pack-objects: teach it to use reachability bitmap index when generating non-stdout pack too

2016-08-08 Thread Kirill Smelkov
o teach `git pack-objects file.pack` to use bitmaps instead of using `git pack-objects --stdout >file.pack` + `git index-pack file.pack`. More context: http://marc.info/?t=146792101400001=1=2 Cc: Vicent Marti <tan...@gmail.com> Helped-by: Jeff King <p...@peff.net> Signed-

Re: [PATCH 1/2] pack-objects: Teach --use-bitmap-index codepath to respect --local, --honor-pack-keep and --incremental

2016-08-08 Thread Kirill Smelkov
( updated patch is in the end of this mail ) Jeff, first of all thanks for commenting, On Mon, Aug 08, 2016 at 09:50:20AM -0400, Jeff King wrote: > On Mon, Aug 08, 2016 at 03:37:35PM +0300, Kirill Smelkov wrote: > > > @@ -958,15 +958,30 @@ static int want_object_in_pack(const u

Re: [PATCH v4 2/2] pack-objects: Teach it to use reachability bitmap index when generating non-stdout pack too

2016-08-08 Thread Kirill Smelkov
On Mon, Aug 08, 2016 at 11:08:34AM -0700, Junio C Hamano wrote: > Kirill Smelkov <k...@nexedi.com> writes: > > > Thanks for the info. I did not knew about show-index when I was starting > > to work on this and later it just came out of sight. Please find > > correct

Re: [PATCH v4 2/2] pack-objects: Teach it to use reachability bitmap index when generating non-stdout pack too

2016-08-08 Thread Kirill Smelkov
On Mon, Aug 08, 2016 at 09:56:00AM -0400, Jeff King wrote: > On Fri, Jul 29, 2016 at 10:47:46AM +0300, Kirill Smelkov wrote: > > > @@ -2527,7 +2528,7 @@ static int get_object_list_from_bitmap(struct > > rev_info *revs) > > if (prepare_bitmap_walk(revs) < 0

Re: [PATCH 1/2] pack-objects: Teach --use-bitmap-index codepath to respect --local, --honor-pack-keep and --incremental

2016-08-08 Thread Kirill Smelkov
On Mon, Aug 01, 2016 at 11:17:30AM -0700, Junio C Hamano wrote: > Kirill Smelkov <k...@nexedi.com> writes: > > > Since 6b8fda2d (pack-objects: use bitmaps when packing objects) there > > are two codepaths in pack-objects: with & without using bitmap > > r

[PATCH v4 2/2] pack-objects: Teach it to use reachability bitmap index when generating non-stdout pack too

2016-07-29 Thread Kirill Smelkov
ion-control.git/299107 http://article.gmane.org/gmane.comp.version-control.git/299420 http://article.gmane.org/gmane.comp.version-control.git/300217 Cc: Vicent Marti <tan...@gmail.com> Helped-by: Jeff King <p...@peff.net> Signed-off-by: Kirill Smelkov <k...@nexedi.com> --- Docum

[PATCH 1/2] pack-objects: Teach --use-bitmap-index codepath to respect --local, --honor-pack-keep and --incremental

2016-07-29 Thread Kirill Smelkov
Since 6b8fda2d (pack-objects: use bitmaps when packing objects) there are two codepaths in pack-objects: with & without using bitmap reachability index. However add_object_entry_from_bitmap(), despite its non-bitmapped counterpart add_object_entry(), in no way does check for whether --local or

Re: [PATCH] pack-objects: Use reachability bitmap index when generating non-stdout pack too

2016-07-29 Thread Kirill Smelkov
On Thu, Jul 28, 2016 at 02:18:29PM -0700, Junio C Hamano wrote: > Kirill Smelkov <k...@nexedi.com> writes: > > > I'm waiting so long for main patch to be at least queued to pu, that I'm > > now a bit frustrated and ready to do something not related to main goal :) >

Re: [PATCH] pack-objects: Use reachability bitmap index when generating non-stdout pack too

2016-07-28 Thread Kirill Smelkov
Junio, first of all thanks for feedback, On Wed, Jul 27, 2016 at 01:40:36PM -0700, Junio C Hamano wrote: > Kirill Smelkov <k...@nexedi.com> writes: > > > > From: Kirill Smelkov <k...@nexedi.com> > > Subject: [PATCH 1/2] pack-objects: Make sure use_bitm

Re: [PATCH] pack-objects: Use reachability bitmap index when generating non-stdout pack too

2016-07-27 Thread Kirill Smelkov
On Mon, Jul 25, 2016 at 02:40:25PM -0400, Jeff King wrote: > On Wed, Jul 13, 2016 at 01:52:17PM +0300, Kirill Smelkov wrote: > > > > So I think if you were to repeatedly "git repack -adb" over time, you > > > would get worse and worse ordering as objects

Re: [PATCH] pack-objects: Use reachability bitmap index when generating non-stdout pack too

2016-07-19 Thread Kirill Smelkov
On Tue, Jul 19, 2016 at 05:29:07AM -0600, Jeff King wrote: > On Sun, Jul 17, 2016 at 08:06:49PM +0300, Kirill Smelkov wrote: > > > > Anyway, please find below updated patch according to your suggestion. > > > Hope it is ok now. > > > > Ping. Is the patch

Re: [PATCH] pack-objects: Use reachability bitmap index when generating non-stdout pack too

2016-07-17 Thread Kirill Smelkov
On Wed, Jul 13, 2016 at 01:52:16PM +0300, Kirill Smelkov wrote: > On Wed, Jul 13, 2016 at 04:26:53AM -0400, Jeff King wrote: > > On Fri, Jul 08, 2016 at 01:38:55PM +0300, Kirill Smelkov wrote: > > > > > > - we will not compute the same write order (which is based on

Re: [PATCH] pack-objects: Use reachability bitmap index when generating non-stdout pack too

2016-07-13 Thread Kirill Smelkov
On Wed, Jul 13, 2016 at 04:26:53AM -0400, Jeff King wrote: > On Fri, Jul 08, 2016 at 01:38:55PM +0300, Kirill Smelkov wrote: > > > > - we will not compute the same write order (which is based on > > > traversal order), leading to packs that h

Re: [PATCH] pack-objects: Use reachability bitmap index when generating non-stdout pack too

2016-07-12 Thread Kirill Smelkov
On Fri, Jul 08, 2016 at 01:38:55PM +0300, Kirill Smelkov wrote: > Peff first of all thanks for feedback, > > On Thu, Jul 07, 2016 at 04:52:23PM -0400, Jeff King wrote: > > On Thu, Jul 07, 2016 at 10:09:17PM +0300, Kirill Smelkov wrote: > > > > > Starting from 6b8fd

Re: [PATCH] pack-objects: Use reachability bitmap index when generating non-stdout pack too

2016-07-08 Thread Kirill Smelkov
Peff first of all thanks for feedback, On Thu, Jul 07, 2016 at 04:52:23PM -0400, Jeff King wrote: > On Thu, Jul 07, 2016 at 10:09:17PM +0300, Kirill Smelkov wrote: > > > Starting from 6b8fda2d (pack-objects: use bitmaps when packing objects) > > if a repository has bitmap index

[PATCH] pack-objects: Use reachability bitmap index when generating non-stdout pack too

2016-07-07 Thread Kirill Smelkov
y packs on repositories restoration. That was my initial motivation for the patch. [1] https://lab.nexedi.com/nexedi/erp5 [2] https://lab.nexedi.com/kirr/git-backup Cc: Vicent Marti <tan...@gmail.com> Cc: Jeff King <p...@peff.net> Signed-off-by: Kirill Smelkov <k...@nexedi.com> --

[ANNOUNCE] Git-backup - Backup set of Git repositories & just files; efficiently

2016-01-27 Thread Kirill Smelkov
https://lab.nexedi.com/kirr/git-backup.git This program backups files and set of bare Git repositories into one Git repository. Files are copied to blobs and then added to tree under certain place, and for Git repositories, all reachable objects are pulled in with maintaining index which

Re: [PATCH 17/19] Portable alloca for Git

2014-04-09 Thread Kirill Smelkov
On Thu, Mar 27, 2014 at 06:22:50PM +0400, Kirill Smelkov wrote: On Mon, Mar 24, 2014 at 02:47:24PM -0700, Junio C Hamano wrote: Kirill Smelkov k...@mns.spb.ru writes: On Fri, Feb 28, 2014 at 06:19:58PM +0100, Erik Faye-Lund wrote: On Fri, Feb 28, 2014 at 6:00 PM, Kirill Smelkov k

Re: [PATCH v2 18/19] tree-diff: rework diff_tree() to generate diffs for multiparent cases as well

2014-04-07 Thread Kirill Smelkov
On Mon, Apr 07, 2014 at 10:29:46AM -0700, Junio C Hamano wrote: Kirill Smelkov k...@navytux.spb.ru writes: The following ... maybe looks a bit simpler, but calls tree_entry_pathcmp twice more times. Besides for important nparent=1 case we were not calling tree_entry_pathcmp at all

Re: [PATCH v2 18/19] tree-diff: rework diff_tree() to generate diffs for multiparent cases as well

2014-04-06 Thread Kirill Smelkov
Junio, First of all thanks a lot for reviewing this patch. I'll reply inline with corrected version attached in the end. On Fri, Apr 04, 2014 at 11:42:39AM -0700, Junio C Hamano wrote: Kirill Smelkov k...@navytux.spb.ru writes: +extern +struct combine_diff_path *diff_tree_paths

Re: [PATCH 17/19] Portable alloca for Git

2014-03-27 Thread Kirill Smelkov
On Mon, Mar 24, 2014 at 02:47:24PM -0700, Junio C Hamano wrote: Kirill Smelkov k...@mns.spb.ru writes: On Fri, Feb 28, 2014 at 06:19:58PM +0100, Erik Faye-Lund wrote: On Fri, Feb 28, 2014 at 6:00 PM, Kirill Smelkov k...@mns.spb.ru wrote: ... In fact that would be maybe preferred

Re: [PATCH v2 18/19] tree-diff: rework diff_tree() to generate diffs for multiparent cases as well

2014-03-27 Thread Kirill Smelkov
On Mon, Feb 24, 2014 at 08:21:50PM +0400, Kirill Smelkov wrote: [...] not changed: - low-level helpers are still named with __ prefix as, imho, that is the best convention to name such helpers, without sacrificing signal/noise ratio. All of them are now static though. Please find

Re: [PATCH v2 16/19] tree-diff: reuse base str(buf) memory on sub-tree recursion

2014-03-27 Thread Kirill Smelkov
On Tue, Mar 25, 2014 at 01:23:20PM +0400, Kirill Smelkov wrote: On Mon, Mar 24, 2014 at 02:43:36PM -0700, Junio C Hamano wrote: Kirill Smelkov k...@mns.spb.ru writes: instead of allocating it all the time for every subtree in __diff_tree_sha1, let's allocate it once in diff_tree_sha1

Re: [PATCH 15/19] tree-diff: no need to call full diff_tree_sha1 from show_path()

2014-03-27 Thread Kirill Smelkov
On Mon, Feb 24, 2014 at 08:21:47PM +0400, Kirill Smelkov wrote: As described in previous commit, when recursing into sub-trees, we can use lower-level tree walker, since its interface is now sha1 based. The change is ok, because diff_tree_sha1() only invokes __diff_tree_sha1(), and also

Re: [PATCH v2 14/19] tree-diff: rework diff_tree interface to be sha1 based

2014-03-27 Thread Kirill Smelkov
On Wed, Mar 26, 2014 at 02:34:24PM -0700, Junio C Hamano wrote: Kirill Smelkov k...@navytux.spb.ru writes: On Tue, Mar 25, 2014 at 10:46:32AM -0700, Junio C Hamano wrote: Kirill Smelkov k...@navytux.spb.ru writes: What are the downsides of __ prefix by the way? Aren't these names

Re: [PATCH v2 14/19] tree-diff: rework diff_tree interface to be sha1 based

2014-03-27 Thread Kirill Smelkov
+stefanbeller On Thu, Mar 27, 2014 at 11:48:11AM -0700, Junio C Hamano wrote: Kirill Smelkov k...@navytux.spb.ru writes: (please keep author email) 8 From: Kirill Smelkov k...@mns.spb.ru Date: Mon, 24 Feb 2014 20:21:46 +0400 Subject: [PATCH v3a] tree-diff: rework diff_tree

Re: [PATCH v2 14/19] tree-diff: rework diff_tree interface to be sha1 based

2014-03-26 Thread Kirill Smelkov
On Tue, Mar 25, 2014 at 10:46:32AM -0700, Junio C Hamano wrote: Kirill Smelkov k...@navytux.spb.ru writes: What are the downsides of __ prefix by the way? Aren't these names reserved for compiler/runtime implementations? Yes, but there are precedents when people don't obey it widely

Re: [PATCH 12/19] tree-diff: remove special-case diff-emitting code for empty-tree cases

2014-03-25 Thread Kirill Smelkov
On Mon, Mar 24, 2014 at 02:18:10PM -0700, Junio C Hamano wrote: Kirill Smelkov k...@mns.spb.ru writes: via teaching tree_entry_pathcmp() how to compare empty tree descriptors: Drop this line, as you explain the pretend empty compares bigger than anything else idea later anyway

Re: [PATCH v2 14/19] tree-diff: rework diff_tree interface to be sha1 based

2014-03-25 Thread Kirill Smelkov
On Mon, Mar 24, 2014 at 02:36:22PM -0700, Junio C Hamano wrote: Kirill Smelkov k...@mns.spb.ru writes: The downside is that try_to_follow_renames(), if active, we cause re-reading of 2 initial trees, which was negligible based on my timings, That would depend on how often the codepath

Re: [PATCH 11/19] tree-diff: simplify tree_entry_pathcmp

2014-03-25 Thread Kirill Smelkov
On Mon, Mar 24, 2014 at 02:25:04PM -0700, Junio C Hamano wrote: Kirill Smelkov k...@mns.spb.ru writes: Since an earlier Finally switch over tree descriptors to contain a pre-parsed entry, we can safely access all tree_desc-entry fields directly instead of first extracting them through

Re: [PATCH v2 16/19] tree-diff: reuse base str(buf) memory on sub-tree recursion

2014-03-25 Thread Kirill Smelkov
On Mon, Mar 24, 2014 at 02:43:36PM -0700, Junio C Hamano wrote: Kirill Smelkov k...@mns.spb.ru writes: instead of allocating it all the time for every subtree in __diff_tree_sha1, let's allocate it once in diff_tree_sha1, and then all callee just use it in stacking style, without memory

Re: [PATCH 17/19] Portable alloca for Git

2014-03-05 Thread Kirill Smelkov
On Fri, Feb 28, 2014 at 06:19:58PM +0100, Erik Faye-Lund wrote: On Fri, Feb 28, 2014 at 6:00 PM, Kirill Smelkov k...@mns.spb.ru wrote: On Fri, Feb 28, 2014 at 02:50:04PM +0100, Erik Faye-Lund wrote: On Fri, Feb 28, 2014 at 2:44 PM, Erik Faye-Lund kusmab...@gmail.com wrote: On Mon, Feb

Re: [PATCH 17/19] Portable alloca for Git

2014-02-28 Thread Kirill Smelkov
On Fri, Feb 28, 2014 at 02:50:04PM +0100, Erik Faye-Lund wrote: On Fri, Feb 28, 2014 at 2:44 PM, Erik Faye-Lund kusmab...@gmail.com wrote: On Mon, Feb 24, 2014 at 5:21 PM, Kirill Smelkov k...@mns.spb.ru wrote: diff --git a/Makefile b/Makefile index dddaf4f..0334806 100644 --- a/Makefile

Re: [PATCH v2 00/19] Multiparent diff tree-walker + combine-diff speedup

2014-02-25 Thread Kirill Smelkov
On Tue, Feb 25, 2014 at 06:43:24AM +0700, Duy Nguyen wrote: On Mon, Feb 24, 2014 at 11:21 PM, Kirill Smelkov k...@mns.spb.ru wrote: Hello up there. Here go combine-diff speedup patches in form of first reworking diff tree-walker to work in general case - when a commit have several parents

[PATCH 02/19] combine-diff: move changed-paths scanning logic into its own function

2014-02-24 Thread Kirill Smelkov
, but faster, paths scanning implementation in the next patch. Signed-off-by: Kirill Smelkov k...@mns.spb.ru Signed-off-by: Junio C Hamano gits...@pobox.com --- ( re-posting without change ) combine-diff.c | 80 ++ 1 file changed, 53 insertions

[PATCH v2 00/19] Multiparent diff tree-walker + combine-diff speedup

2014-02-24 Thread Kirill Smelkov
. Signoffs were left intact, if a patch was already applied to pu with one, and had not changed. Please apply and thanks, Kirill P.S. Sorry for the delay - I was very busy. Kirill Smelkov (19): combine-diff: move show_log_first logic/action out of paths scanning combine-diff: move changed

[PATCH 01/19] combine-diff: move show_log_first logic/action out of paths scanning

2014-02-24 Thread Kirill Smelkov
care to explicitly not show anything if parents array is empty, as in fact there are some clients in Git code, which calls diff_tree_combined() in such a way. Signed-off-by: Kirill Smelkov k...@mns.spb.ru Signed-off-by: Junio C Hamano gits...@pobox.com --- ( re-posting without change

[PATCH 13/19] tree-diff: diff_tree() should now be static

2014-02-24 Thread Kirill Smelkov
We reworked all its users to use the functionality through diff_tree_sha1 variant in recent patches (see tree-diff: allow diff_tree_sha1 to accept NULL sha1 and what comes next). diff_tree() is now not used outside tree-diff.c - make it static. Signed-off-by: Kirill Smelkov k...@mns.spb.ru

[PATCH 09/19] tree-diff: rename compare_tree_entry - tree_entry_pathcmp

2014-02-24 Thread Kirill Smelkov
to it. Signed-off-by: Kirill Smelkov k...@mns.spb.ru Signed-off-by: Junio C Hamano gits...@pobox.com --- ( re-posting without change ) tree-diff.c | 15 +-- 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/tree-diff.c b/tree-diff.c index 6207372..3345534 100644 --- a/tree-diff.c

[PATCH 11/19] tree-diff: simplify tree_entry_pathcmp

2014-02-24 Thread Kirill Smelkov
-by: Kirill Smelkov k...@mns.spb.ru Signed-off-by: Junio C Hamano gits...@pobox.com --- ( re-posting without change ) tree-diff.c | 17 ++--- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/tree-diff.c b/tree-diff.c index 20a4fda..cf96ad7 100644 --- a/tree-diff.c +++ b

[PATCH 06/19] tree-diff: consolidate code for emitting diffs and recursion in one place

2014-02-24 Thread Kirill Smelkov
the same. Signed-off-by: Kirill Smelkov k...@mns.spb.ru Signed-off-by: Junio C Hamano gits...@pobox.com --- ( re-posting without change ) tree-diff.c | 115 1 file changed, 84 insertions(+), 31 deletions(-) diff --git a/tree-diff.c b

[PATCH 12/19] tree-diff: remove special-case diff-emitting code for empty-tree cases

2014-02-24 Thread Kirill Smelkov
iterated simultaneously, and some pair of them would run to the end. Signed-off-by: Kirill Smelkov k...@mns.spb.ru Signed-off-by: Junio C Hamano gits...@pobox.com --- ( re-posting without change ) tree-diff.c | 21 + 1 file changed, 9 insertions(+), 12 deletions(-) diff --git

[PATCH 04/19] tree-diff: no need to pass match to skip_uninteresting()

2014-02-24 Thread Kirill Smelkov
It is neither used there as input, nor the output written through it, is used outside. Signed-off-by: Kirill Smelkov k...@mns.spb.ru Signed-off-by: Junio C Hamano gits...@pobox.com --- ( re-posting without change ) tree-diff.c | 17 - 1 file changed, 8 insertions(+), 9

[PATCH 19/19] combine-diff: speed it up, by using multiparent diff tree-walker directly

2014-02-24 Thread Kirill Smelkov
stayed the same. Signed-off-by: Kirill Smelkov k...@mns.spb.ru Signed-off-by: Junio C Hamano gits...@pobox.com --- ( re-posting without change ) combine-diff.c | 88 ++ diff.c | 1 + 2 files changed, 84 insertions(+), 5 deletions

[PATCH 15/19] tree-diff: no need to call full diff_tree_sha1 from show_path()

2014-02-24 Thread Kirill Smelkov
, as we have added a path and '/' before recursing. Signed-off-by: Kirill Smelkov k...@mns.spb.ru Signed-off-by: Junio C Hamano gits...@pobox.com --- ( re-posting without change ) tree-diff.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tree-diff.c b/tree-diff.c index

[PATCH 17/19] Portable alloca for Git

2014-02-24 Thread Kirill Smelkov
Cc: Petr Salinger petr.salin...@seznam.cz Cc: Jonathan Nieder jrnie...@gmail.com Cc: Thomas Schwinge tschwi...@gnu.org Signed-off-by: Kirill Smelkov k...@mns.spb.ru --- ( new patch ) Makefile | 6 ++ config.mak.uname | 10 -- configure.ac | 8 git-compat

[PATCH 03/19] tree-diff: no need to manually verify that there is no mode change for a path

2014-02-24 Thread Kirill Smelkov
-tree recursion), which pre-dates base_name_compare() introduction in 958ba6c9 (Introduce base_name_compare() helper function) by a month. Signed-off-by: Kirill Smelkov k...@mns.spb.ru Signed-off-by: Junio C Hamano gits...@pobox.com --- ( re-posting without change ) tree-diff.c | 15

[PATCH 05/19] tree-diff: show_tree() is not needed

2014-02-24 Thread Kirill Smelkov
. Signed-off-by: Kirill Smelkov k...@mns.spb.ru --- ( re-posting without change ) tree-diff.c | 35 +++ 1 file changed, 3 insertions(+), 32 deletions(-) diff --git a/tree-diff.c b/tree-diff.c index a8c2aec..2ad7788 100644 --- a/tree-diff.c +++ b/tree-diff.c @@ -55,25

[PATCH v2 18/19] tree-diff: rework diff_tree() to generate diffs for multiparent cases as well

2014-02-24 Thread Kirill Smelkov
separate changes to a file), or an evil one, the map will not be full, i.e. some valid sha1 would be absent from it. That case was my initial motivation for combined diffs speedup. Signed-off-by: Kirill Smelkov k...@mns.spb.ru --- Changes since v1: - fixed last-minute thinko/bug last time

[PATCH 10/19] tree-diff: show_path prototype is not needed anymore

2014-02-24 Thread Kirill Smelkov
We moved all action-taking code below show_path() in recent HEAD~~ (tree-diff: move all action-taking code out of compare_tree_entry). Signed-off-by: Kirill Smelkov k...@mns.spb.ru Signed-off-by: Junio C Hamano gits...@pobox.com --- ( re-posting without change ) tree-diff.c | 3 --- 1 file

[PATCH 08/19] tree-diff: move all action-taking code out of compare_tree_entry()

2014-02-24 Thread Kirill Smelkov
it to was-already-there same switch/if, so the result is maybe a little bit faster. Signed-off-by: Kirill Smelkov k...@mns.spb.ru Signed-off-by: Junio C Hamano gits...@pobox.com --- ( re-posting without change ) tree-diff.c | 28 1 file changed, 12 insertions(+), 16 deletions

[PATCH 07/19] tree-diff: don't assume compare_tree_entry() returns -1,0,1

2014-02-24 Thread Kirill Smelkov
for multiparent walker and will stay that way. =0 case goes first, because it happens more often in real diffs - i.e. paths are the same. Signed-off-by: Kirill Smelkov k...@mns.spb.ru Signed-off-by: Junio C Hamano gits...@pobox.com --- ( re-posting without change ) tree-diff.c | 22

[PATCH v2 16/19] tree-diff: reuse base str(buf) memory on sub-tree recursion

2014-02-24 Thread Kirill Smelkov
--raw --no-abbrev --no-renames --format='%H' navy.gitlinux.git v3.10..v3.11 before 0.618s 1.903s after 0.611s 1.889s speedup 1.1%0.7% Signed-off-by: Kirill Smelkov k...@mns.spb.ru --- Changes since v1: - don't need to touch

Re: [PATCH 1/2] tree-diff: rework diff_tree() to generate diffs for multiparent cases as well

2014-02-16 Thread Kirill Smelkov
On Fri, Feb 14, 2014 at 09:37:00AM -0800, Junio C Hamano wrote: Kirill Smelkov k...@mns.spb.ru writes: Previously diff_tree(), which is now named __diff_tree_sha1(), was That name with two leading underscores is a rather unfortunate, especially for a function that is not a file scope

Re: [PATCH 2/2] combine-diff: speed it up, by using multiparent diff tree-walker directly

2014-02-14 Thread Kirill Smelkov
On Thu, Feb 13, 2014 at 11:55:08AM -0800, Junio C Hamano wrote: Kirill Smelkov k...@mns.spb.ru writes: + if (need_generic_pathscan) { + /* NOTE generic case also handles --stat, as it computes +* diff(sha1,parent_i) for all i to do the job, specifically

Re: [PATCH 1/2] tree-diff: rework diff_tree() to generate diffs for multiparent cases as well

2014-02-14 Thread Kirill Smelkov
On Thu, Feb 13, 2014 at 11:51:19AM -0800, Junio C Hamano wrote: Kirill Smelkov k...@mns.spb.ru writes: + /* until we go to it next round, .next holds how many bytes we +* allocated (for faster realloc - we don't need copying old data

Re: [PATCH 11/11] tree-diff: reuse base str(buf) memory on sub-tree recursion

2014-02-13 Thread Kirill Smelkov
On Fri, Feb 07, 2014 at 09:48:52PM +0400, Kirill Smelkov wrote: instead of allocating it all the time for every subtree in __diff_tree_sha1, let's allocate it once in diff_tree_sha1, and then all callee just use it in stacking style, without memory allocations. This should be faster

[PATCH 0/2] Multiparent diff tree-walker + combine-diff speedup

2014-02-13 Thread Kirill Smelkov
, which for linux.git results in ~14 times speedup. I understand v1.9.0 is going to be released first, but wanted to finally send the patches, so that people could start reviewing them. Please apply on top of ks/tree-diff-more and thanks beforehand, Kirill Kirill Smelkov (2): tree-diff: rework

[PATCH 2/2] combine-diff: speed it up, by using multiparent diff tree-walker directly

2014-02-13 Thread Kirill Smelkov
The result stayed the same. Signed-off-by: Kirill Smelkov k...@mns.spb.ru --- combine-diff.c | 85 ++ diff.c | 1 + 2 files changed, 81 insertions(+), 5 deletions(-) diff --git a/combine-diff.c b/combine-diff.c index 1732dfd..ddf7495

[PATCH 1/2] tree-diff: rework diff_tree() to generate diffs for multiparent cases as well

2014-02-13 Thread Kirill Smelkov
motivation for combined diffs speedup. Signed-off-by: Kirill Smelkov k...@mns.spb.ru --- diff.c | 1 + diff.h | 10 ++ tree-diff.c | 501 3 files changed, 448 insertions(+), 64 deletions(-) diff --git a/diff.c b/diff.c index

Re: [PATCH 00/11] More preparatory work for multiparent tree-walker

2014-02-13 Thread Kirill Smelkov
On Wed, Feb 12, 2014 at 09:25:51AM -0800, Junio C Hamano wrote: Junio C Hamano gits...@pobox.com writes: Kirill Smelkov k...@mns.spb.ru writes: Sorry for the confusion. Could you please do the following: Patches should be applied over to ks/tree-diff-walk (74aa4a18). Before applying

Re: [PATCH 02/11] tree-diff: consolidate code for emitting diffs and recursion in one place

2014-02-13 Thread Kirill Smelkov
On Thu, Feb 13, 2014 at 09:43:27AM -0800, Junio C Hamano wrote: Kirill Smelkov k...@mns.spb.ru writes: +static void show_path(struct strbuf *base, struct diff_options *opt, + struct tree_desc *t1, struct tree_desc *t2) { unsigned mode; const char *path

Re: [PATCH 00/11] More preparatory work for multiparent tree-walker

2014-02-12 Thread Kirill Smelkov
On Tue, Feb 11, 2014 at 11:59:02AM -0800, Junio C Hamano wrote: Kirill Smelkov k...@mns.spb.ru writes: Sorry for the confusion. Could you please do the following: Patches should be applied over to ks/tree-diff-walk (74aa4a18). Before applying the patches, please cherry-pick

Re: [PATCH 00/11] More preparatory work for multiparent tree-walker

2014-02-11 Thread Kirill Smelkov
On Mon, Feb 10, 2014 at 04:28:33PM -0800, Junio C Hamano wrote: Kirill Smelkov k...@mns.spb.ru writes: Here I'm preparing tree-diff.c to be ready for the new tree-walker, so that the final change is tractable and looks good and non noisy. Some small speedups are gained along the way

[PATCH 00/11] More preparatory work for multiparent tree-walker

2014-02-07 Thread Kirill Smelkov
Kirill Smelkov (11): tree-diff: show_tree() is not needed tree-diff: consolidate code for emitting diffs and recursion in one place tree-diff: don't assume compare_tree_entry() returns -1,0,1 tree-diff: move all action-taking code out of compare_tree_entry() tree-diff: rename

[PATCH 03/11] tree-diff: don't assume compare_tree_entry() returns -1,0,1

2014-02-07 Thread Kirill Smelkov
for multiparent walker and will stay that way. =0 case goes first, because it happens more often in real diffs - i.e. paths are the same. Signed-off-by: Kirill Smelkov k...@mns.spb.ru --- tree-diff.c | 22 ++ 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/tree

[PATCH 10/11] tree-diff: no need to call full diff_tree_sha1 from show_path()

2014-02-07 Thread Kirill Smelkov
, as we have added a path and '/' before recursing. Signed-off-by: Kirill Smelkov k...@mns.spb.ru --- tree-diff.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tree-diff.c b/tree-diff.c index dd6c760..e385ed4 100644 --- a/tree-diff.c +++ b/tree-diff.c @@ -116,8 +116,8

[PATCH 09/11] tree-diff: rework diff_tree interface to be sha1 based

2014-02-07 Thread Kirill Smelkov
be renamed once more later, when its semantic will change to just generate paths for a diff, instead of producing it. So the current name is appropriate, but probably temporary. Signed-off-by: Kirill Smelkov k...@mns.spb.ru --- diff.h | 4 ++-- tree-diff.c | 60

[PATCH 11/11] tree-diff: reuse base str(buf) memory on sub-tree recursion

2014-02-07 Thread Kirill Smelkov
--raw --no-abbrev --no-renames` navy.gitlinux.git v3.10..v3.11 before 0.547s 1.791s after 0.541s 1.777s speedup 1.1%0.8% Signed-off-by: Kirill Smelkov k...@mns.spb.ru --- diff.h | 2 +- tree-diff.c | 36

[PATCH 08/11] tree-diff: remove special-case diff-emitting code for empty-tree cases

2014-02-07 Thread Kirill Smelkov
iterated simultaneously, and some pair of them would run to the end. Signed-off-by: Kirill Smelkov k...@mns.spb.ru --- tree-diff.c | 21 + 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/tree-diff.c b/tree-diff.c index 330ca07..7688402 100644 --- a/tree-diff.c

[PATCH 07/11] tree-diff: simplify tree_entry_pathcmp

2014-02-07 Thread Kirill Smelkov
-by: Kirill Smelkov k...@mns.spb.ru --- tree-diff.c | 17 ++--- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/tree-diff.c b/tree-diff.c index 604dc57..330ca07 100644 --- a/tree-diff.c +++ b/tree-diff.c @@ -15,18 +15,13 @@ */ static int tree_entry_pathcmp(struct tree_desc

  1   2   >