Copying from one repository to another, flattening the early part of the source repo's history in the target repo?

2010-04-29 Thread Dan Stromberg
I'm working on some Subversion-related software for propagating trunk
checkins to any of dozens of target branches, based on trunk's checkin
comments (that is, where the checkins need to subsequently go automatically is
determined by the checkin comments in trunk, and those comments are
prepared pretty consistently by a custom GUI; the GUI piece is already
written and in production).

I of course want it to work well when we deploy it - and the trunk
we'll ultimately be running it against is rather huge.

So to make it work well from the start, I need to test it well.  To
test it, I really want to copy from one SVN server's trunk to another
SVN server's repository - flattening a significant part of the early
history from the source trunk, and then having as-identical-as-practical
checkin comments for the later history.

Is there already a way of setting up such a test branch?  I've googled
about it quite a bit and come up with things that are close (svnsync:
way too much test data, svn merge: too few checkin comments unless I
do them individually somehow), but not quite what I need.  It's
seeming like writing the program to prepare the test environment is
nearly as complex as what I'm intending to test.

Please note that although I've been an SVN admin in the past, I do not
have administrative access to these SVN servers.

TIA!

(PS: I've written a bunch of unit tests for this, but that's just not
enough - it needs to be tested as a complete system too with
minimal deviation from how it's to be used In Real Life)


Re: Copying from one repository to another, flattening the early part of the source repo's history in the target repo?

2010-04-29 Thread Dan Stromberg
On Thu, Apr 29, 2010 at 11:23 AM, Les Mikesell lesmikes...@gmail.com wrote:
 On 4/29/2010 1:06 PM, Dan Stromberg wrote:

 I'm working on some Subversion-related software for propagating trunk
 checkins to any of dozens of target branches, based on trunk's checkin
 comments (that is, where the checkins need to subsequently go
 automatically is
 determined by the checkin comments in trunk, and those comments are
 prepared pretty consistently by a custom GUI; the GUI piece is already
 written and in production).

 I of course want it to work well when we deploy it - and the trunk
 we'll ultimately be running it against is rather huge.

 So to make it work well from the start, I need to test it well.  To
 test it, I really want to copy from one SVN server's trunk to another
 SVN server's repository - flattening a significant part of the early
 history from the source trunk, and then having as-identical-as-practical
 checkin comments for the later history.

 I don't understand the need to flatten the trunk.  Fanning out from the
 trunk to many branches should be straightforward enough, fhough.

The source branch is terabytes of source code.  I want to restrict by
subdirectory, and flatten history to reduce the storage requirements.

As far as straightforward enough, yes, it's conceptually simple enough,
though I've got 600+ lines of Python code written for this so far (not
including unit tests or test inputs), and it's not done yet.  It includes XML
parsing, YAML parsing, SVN interaction (with error checking), a flat file
database module (because the customer doesn't want to use gdbm or
similar), etc.

 Is there already a way of setting up such a test branch?  I've googled
 about it quite a bit and come up with things that are close (svnsync:
 way too much test data, svn merge: too few checkin comments unless I
 do them individually somehow), but not quite what I need.  It's
 seeming like writing the program to prepare the test environment is
 nearly as complex as what I'm intending to test.

 Please note that although I've been an SVN admin in the past, I do not
 have administrative access to these SVN servers.

 The only way you can modify existing history is with a svnadmin
 dump/filter/load and you need admin access for that.   Can you svnsync the
 live repo to one you control, then dump from there to get a base for your
 test?

I guess I'm not entirely looking to modify existing history, so much as compress
the old stuff by flattening and restricting it, and then cloning the
recent history.
I can imagine a program to do this (at least to an extent) that doesn't require
admin access, but I'm hoping someone's already written it.

It's starting to sound like I'm going to need to write a bash script
to handle this.