Re: git-p4 out of memory for very large repository

2013-09-06 Thread Corey Thompson
On Mon, Sep 02, 2013 at 08:42:36PM +0100, Luke Diamand wrote: I guess you could try changing the OOM score for git-fast-import. change /proc/pid/oomadj. I think a value of -31 would make it very unlikely to be killed. On 29/08/13 23:46, Pete Wyckoff wrote: I usually just do git p4 sync

Re: git-p4 out of memory for very large repository

2013-08-28 Thread Corey Thompson
On Mon, Aug 26, 2013 at 09:47:56AM -0400, Corey Thompson wrote: You are correct that git-fast-import is killed by the OOM killer, but I was unclear about which process was malloc()ing so much memory that the OOM killer got invoked (as other completely unrelated processes usually also get

Re: git-p4 out of memory for very large repository

2013-08-26 Thread Corey Thompson
On Sun, Aug 25, 2013 at 11:50:01AM -0400, Pete Wyckoff wrote: Modern git, including your version, do streaming reads from p4, so the git-p4 python process never even holds a whole file's worth of data. You're seeing git-fast-import die, it seems. It will hold onto the entire file contents.

Re: git-p4 out of memory for very large repository

2013-08-23 Thread Corey Thompson
On Fri, Aug 23, 2013 at 08:16:58AM +0100, Luke Diamand wrote: On 23/08/13 02:12, Corey Thompson wrote: Hello, Has anyone actually gotten git-p4 to clone a large Perforce repository? Yes. I've cloned repos with a couple of Gig of files. I have one codebase in particular that gets

Re: git-p4 out of memory for very large repository

2013-08-23 Thread Corey Thompson
On Fri, Aug 23, 2013 at 07:48:56AM -0400, Corey Thompson wrote: Sorry, I guess I could have included more details in my original post. Since then, I have also made an attempt to clone another (slightly more recent) branch, and at last had success. So I see this does indeed work, it just seems

Re: git-p4 out of memory for very large repository

2013-08-23 Thread Corey Thompson
On Fri, Aug 23, 2013 at 08:42:44PM +0100, Luke Diamand wrote: I think I've cloned files as large as that or larger. If you just want to clone this and move on, perhaps you just need a bit more memory? What's the size of your physical memory and swap partition? Per process memory limit? The

git-p4 out of memory for very large repository

2013-08-22 Thread Corey Thompson
Hello, Has anyone actually gotten git-p4 to clone a large Perforce repository? I have one codebase in particular that gets to about 67%, then consistently gets get-fast-import (and often times a few other processes) killed by the OOM killer. I've found some patches out there that claim to