Re: fast-import's gfi_unpack_entry causes too many munmap/mmap cycles

2016-04-16 Thread Mike Hommey
On Sun, Apr 17, 2016 at 09:54:43AM +0900, Mike Hommey wrote: > On Sat, Apr 16, 2016 at 08:04:03PM +0900, Mike Hommey wrote: > > So I think I got myself a workaround... > > > > > A --- B > > > \ > > > \-- C > > > > > > I have: > > > - diff between null-tree and A > > > - diff between A and B >

Re: fast-import's gfi_unpack_entry causes too many munmap/mmap cycles

2016-04-16 Thread Mike Hommey
On Sat, Apr 16, 2016 at 08:04:03PM +0900, Mike Hommey wrote: > So I think I got myself a workaround... > > > A --- B > > \ > > \-- C > > > > I have: > > - diff between null-tree and A > > - diff between A and B > > - diff between B and C > > I should be able to do: > > - start the commit

Re: fast-import's gfi_unpack_entry causes too many munmap/mmap cycles

2016-04-16 Thread Mike Hommey
On Sat, Apr 16, 2016 at 06:18:39PM +0900, Mike Hommey wrote: > And even if I am okay with that overhead, I still hit the problem again > when using that tree later with `M 04 $sha1 ` in the next commit, > because that does a load_tree() for the tree sha1, after a commit having > occurred,

Re: fast-import's gfi_unpack_entry causes too many munmap/mmap cycles

2016-04-16 Thread Mike Hommey
On Sat, Apr 16, 2016 at 06:18:39PM +0900, Mike Hommey wrote: > Now, while each individual case could be improved to avoid > gfi_unpack_entry, it seems to me it would be better to make > gfi_unpack_entry better somehow. Come to think of it, there are cases that might still be worth fixing. Like

fast-import's gfi_unpack_entry causes too many munmap/mmap cycles

2016-04-16 Thread Mike Hommey
Hi, I think I've mentioned this issue in the past, and fixed one case of munmap/mmap cycle in fast-import. I've found more cases where this can happen, and while on Linux, it's not a problem, on OSX, it leads to catastrophic performance, when the import is massive. One part of the problem is