Re: Problem Loading Huge Repository

2011-06-17 Thread Daniel Shahaf
Thomas Harold wrote on Fri, Jun 17, 2011 at 11:50:45 -0400: > On 6/17/2011 10:54 AM, Daniel Shahaf wrote: > >Thomas Harold wrote on Fri, Jun 17, 2011 at 10:31:43 -0400: > >>And if you have a choice of file systems for the repository to be stored > >>on, make sure that it's something which can deal

Re: Problem Loading Huge Repository

2011-06-17 Thread Thomas Harold
On 6/17/2011 10:54 AM, Daniel Shahaf wrote: Thomas Harold wrote on Fri, Jun 17, 2011 at 10:31:43 -0400: And if you have a choice of file systems for the repository to be stored on, make sure that it's something which can deal with a few hundred thousand tiny files. On Linux, I'd suggest going w

Re: Problem Loading Huge Repository

2011-06-17 Thread Daniel Shahaf
Thomas Harold wrote on Fri, Jun 17, 2011 at 10:31:43 -0400: > On 6/16/2011 7:05 PM, Bruno Antunes wrote: > > > >Do you know any faster way to load the dump file or to filter out > >some projects/revisions so I can speed up the process? > > > > Are you CPU-bound? Or are you limited by disk speed? I

Re: Problem Loading Huge Repository

2011-06-17 Thread Thomas Harold
On 6/16/2011 7:05 PM, Bruno Antunes wrote: Do you know any faster way to load the dump file or to filter out some projects/revisions so I can speed up the process? Are you CPU-bound? Or are you limited by disk speed? If you're limited by disk access times, make sure that the source file that

Re: Problem Loading Huge Repository

2011-06-17 Thread Mark Phippard
On Thu, Jun 16, 2011 at 8:08 PM, Bruno Antunes wrote: > Just a thought... Do you need the revision history or only the current > (head) revision? > Guessing if you do not need the revision history then it will be much > smaller and  faster to svn export their-stuff -r HEAD > > I need the entire r

Re: Problem Loading Huge Repository

2011-06-17 Thread kmradke
Bruno Antunes wrote on 06/16/2011 06:05:13 PM: > > As part of the work of my PhD thesis I need to load the ASF > Subversion repository into my own local repository in order to mine > and extract information from the repository without overloading the > ASF servers. > > I have downloaded the r

Re: Problem Loading Huge Repository

2011-06-16 Thread Geoff Hoffman
Um... separate revision range at a time... -r 6000:HEAD ? only individual directories at a time... path/to/dir -r 6000:HEAD ? Or... This is a bit out there but you could spin up and install subversion on a 4XL amazon EC2 instance, import it on this very, very fast (virtual) machine, then the m

Re: Problem Loading Huge Repository

2011-06-16 Thread Bruno Antunes
On Jun 17, 2011, at 24:59 , Geoff Hoffman wrote: > > > On Thu, Jun 16, 2011 at 4:05 PM, Bruno Antunes wrote: > Hi, > > As part of the work of my PhD thesis I need to load the ASF Subversion > repository into my own local repository in order to mine and extract > information from the reposit

Re: Problem Loading Huge Repository

2011-06-16 Thread Geoff Hoffman
On Thu, Jun 16, 2011 at 4:05 PM, Bruno Antunes wrote: > Hi, > > As part of the work of my PhD thesis I need to load the ASF Subversion > repository into my own local repository in order to mine and extract > information from the repository without overloading the ASF servers. > > I have downloade

Problem Loading Huge Repository

2011-06-16 Thread Bruno Antunes
Hi, As part of the work of my PhD thesis I need to load the ASF Subversion repository into my own local repository in order to mine and extract information from the repository without overloading the ASF servers. I have downloaded the repository dump and started loading it into my own reposito