Re: cherry-pick very slow on big repository

2017-11-21 Thread Elijah Newren
On Tue, Nov 21, 2017 at 4:07 AM, Peter Krefting wrote: > Elijah Newren: > >> Sure, take a look at the big-repo-small-cherry-pick branch of >> https://github.com/newren/git > > > With those changes, the time usage is the same as if I set > merge.renameLimit=1 for the

Re: cherry-pick very slow on big repository

2017-11-21 Thread Peter Krefting
Elijah Newren: Sure, take a look at the big-repo-small-cherry-pick branch of https://github.com/newren/git With those changes, the time usage is the same as if I set merge.renameLimit=1 for the repository, and the end result is identical: $ time

Re: cherry-pick very slow on big repository

2017-11-13 Thread Elijah Newren
On Mon, Nov 13, 2017 at 3:22 AM, Peter Krefting wrote: > Elijah Newren: > >> I would be very interested to hear how my rename detection performance >> patches work for you; this kind of usecase was the exact one it was designed >> to help the most. See >>

Re: cherry-pick very slow on big repository

2017-11-13 Thread Peter Krefting
Elijah Newren: I would be very interested to hear how my rename detection performance patches work for you; this kind of usecase was the exact one it was designed to help the most. See https://public-inbox.org/git/20171110222156.23221-1-new...@gmail.com/ I'd be happy to try them out. Is

RE: cherry-pick very slow on big repository

2017-11-13 Thread Peter Krefting
Kevin Willford: Since this is happening during a merge, you might need to use merge.renameLimit or the merge strategy option of -Xno-renames. Although the code does fallback to use the diff.renameLimit but there is still a lot that is done before even checking the rename limit so I would

Re: cherry-pick very slow on big repository

2017-11-10 Thread Elijah Newren
On Fri, Nov 10, 2017 at 6:05 AM, Peter Krefting wrote: > Derrick Stolee: > >> Git is spending time detecting renames, which implies you probably renamed >> a folder or added and deleted a large number of files. This rename detection >> is quadratic (# adds times #

Re: cherry-pick very slow on big repository

2017-11-10 Thread Elijah Newren
Interesting timing. I have some performance patches specifically developed because rename detection during merges made a small cherry-pick in a large repo rather slow...in my case, I dropped the time for the cherry pick by a factor of about 30 (no guarantees you'll see the same; it's very

RE: cherry-pick very slow on big repository

2017-11-10 Thread Kevin Willford
net>; Git Mailing List <git@vger.kernel.org> > Subject: Re: cherry-pick very slow on big repository > > Derrick Stolee: > > > Git is spending time detecting renames, which implies you probably > > renamed a folder or added and deleted a large number of files. Th

Re: cherry-pick very slow on big repository

2017-11-10 Thread Peter Krefting
Derrick Stolee: Git is spending time detecting renames, which implies you probably renamed a folder or added and deleted a large number of files. This rename detection is quadratic (# adds times # deletes). Yes, a couple of directories with a lot of template files have been renamed (and

Re: cherry-pick very slow on big repository

2017-11-10 Thread Derrick Stolee
On 11/10/2017 7:37 AM, Peter Krefting wrote: Jeff King: Can you get a backtrace? I'd do something like: Seems that it spends most time in diffcore_count_changes(), that is where it hits whenever I hit Ctrl+C (various line numbers 199-207 in diffcore-delta.c; this is on the v2.15.0 tag).

Re: cherry-pick very slow on big repository

2017-11-10 Thread Peter Krefting
Jeff King: Can you get a backtrace? I'd do something like: Seems that it spends most time in diffcore_count_changes(), that is where it hits whenever I hit Ctrl+C (various line numbers 199-207 in diffcore-delta.c; this is on the v2.15.0 tag). (gdb) bt #0 diffcore_count_changes

Re: cherry-pick very slow on big repository

2017-11-10 Thread Jeff King
On Fri, Nov 10, 2017 at 10:39:39AM +0100, Peter Krefting wrote: > Running strace, it seems like it is doing lstat(), open(), mmap(), close() > and munmap() on every single file in the repository, which takes a lot of > time. > > I thought it was just updating the status, but "git status" returns

cherry-pick very slow on big repository

2017-11-10 Thread Peter Krefting
Hi! On a big repository (57000 files, 2,5 gigabytes in .git/objects), git cherry-pick is very slow for me (v2.15.0). This is cherry-picking a one-file change, where the file is in the same place on both branches, and which applies cleanly (I am backporting a few fixes to a maintenance