date:20100802

From: Les Mikesell [mailto:lesmikes...@gmail.com] 
> On 8/2/2010 3:46 PM, Vallon, Justin wrote:
> > That means filesystem snapshots pass the transactional test.
> 
> Maybe - is there a guarantee that the app flushes to disk in the 
> expected order?  Or do snapshots take the current dirty filesystem 
> buffers into account?

If the client or server filesystem buffers are dirty, then the application has 
not flushed the data.  Therefore, the application should make no assumption 
about whether the data has been written through to its final media.  What is 
critical is that (a) everything gets flushed, then (b) current is updated, then 
(c) current is flushed.

This generalizes to:

1) Do whatever you want so long as it can be discarded and/or be written out of 
order
2) Flush everything
3) Write a flag saying that everything has been committed
4) Flush again

On the topic of whether dirty server-side buffers are in the snapshot, there 
are two reasons the data could be dirty:

a) The client is in the middle of writing some data, the server is about to 
write it, and the client is waiting.
b) The client wrote the data asynchronously, and the server can write it 
whenever it wants (async write).

In either case, it would not be surprising for the data to be included in the 
snapshot or excluded.  However, in neither case should the client have expected 
that the data be present in the snapshot, since it was either in-progress or a 
delayed write.

All of this can get thrown out the window if the server decides to "lie".  See 
http://nfs.sourceforge.net/nfs-howto/ar01s05.html, section 5.9, where it 
describes how the Linux async NFS export option can cause the NFS server to lie 
when asked to fsync or flush-on-close.  The problem here is that the server 
might keep the revs (part 1) in cache, while writing out the current file (part 
3), leaving a corrupted repository if the server fails.  So, this is a problem 
in general, not specific to filesystem snapshots.

-- 
-Justin

Re: Support for filesystem snapshots (?)

2010-08-02 Thread Blair Zajac

On 08/02/2010 01:27 PM, Stefan Sperling wrote:

On Mon, Aug 02, 2010 at 03:25:48PM -0400, Vallon, Justin wrote:

E.g. Subversion's FSFS needs to create a revision file from the commit's
transaction, and move the finalized revision file into place.
After the revision file has been moved into place successfully, FSFS also
updates the svn:date revision property and moves the revision properties file
into place (or copies revprop data into an sqlite database if you use
revprop packing). Then, it updates the 'current' file which contains the
number of the current HEAD revision. If you use representation sharing to
save disk space, the commit may involve further updates to yet another
sqlite database.

All these actions need to complete in order to have a consistent state.

If you're interested in seeing the code that does this, look at the
svn_fs_fs__commit() and commit_body() functions in
http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/fs_fs.c

I see this is executed with a FS write lock. My concern would be
focused on the interaction between the commit code and any rollback
code. For example, if the commit dies (any any point during the
commit), what will be required to insure that the repository behaves
as if the commit never started? Will a repo cleanup be required; will
the next committer cleanup the partial rev automatically (ie:
overwrite stale files); will the repo be hopelessly inconsistent?

I honestly didn't know so I went and asked.
And learned something!

users asking interesting questions:
http://mail-archives.apache.org/mod_mbox/subversion-users/201008.mbox/%3c6ec02a00cc9f684daf4af4084ca84d5f01c40...@drmbx3.winmail.deshaw.com%3e
i dunno how fsfs behaves in face of an interrupted commit; whether or
not it needs to be rolled back
if you haven't touched current than the rev file will never be read
and will be overwritten
stsp: does that answer your question?
i think so
because the rev file of the following commit will have the same name to
move things into place onto
write lock only for revprop change and commit
:-)
so, using rsync for backup is fine?
if you copy current first, yes
what's hotcopy for then? just bdb?
stsp: copying 'current' first ... :-)

ok, so what happens if I don't copy current first?
you can copy revs/
then a commit happens
then you copy current
so you don't have all of revs/ that current claims exist
then I need to unwedge it
by decrementing current
right
and hopefully you haven't just crossed a packing boundary
eg if you want to decrement from 1002 to 999
and someone packed it already
a bit more work

So in the event that 'current' says you are at rN but the rev data in the
repository is still at r'N-1', the repository will complain (I've tried
that, "No such revision rN"), and you'll need to decrement the counter
in 'current'. But otherwise, the repository will continue to work.

Now, how does rsync, or a file-system snapshot, know to make sure that
'current' is always copied first? Even if you copy 'current' first manually,
rsync might later overwrite it. But unless you use packing it's trivial to
fix the backup if it breaks, and all you risk is losing the most recent HEAD
revision, which you may not have gotten with a hotcopy anyway.

Still, I think I'll keep advising people to use hotcopy.
It avoids the problem with a too recent 'current' file, i.e. the backup
is always usable out of the box. And who knows how Subversion's on-disk
formats will at change in the future.
The hotcopy approach will always be supported, and works fine if, as you
pointed out, you can make sure that a hotcopy is being backup up while
not being written to.

There's this tool that should order rsync correctly:

http://svn.apache.org/repos/asf/subversion/trunk/contrib/server-side/svn-fast-backup

But in general, you're correct, with the additions to the fsfs
repository, this script is not always kept up to date.

Blair

Re: Moving NAS owner and upgrading?

2010-08-02 Thread Christopher Nagel

Thanks Andy, especially for the hint about stopping svnserve!

I think I'll let the existing repository's mounted space go back into the pool 
and just reserve a new one for the new repos, this way both can be active at 
the 
same time.  That'll make things more straightforward.

Best regards,
Chris

From: Andy Levy 
To: Christopher Nagel 
Cc: users@subversion.apache.org
Sent: Mon, August 2, 2010 9:22:18 AM
Subject: Re: Moving NAS owner and upgrading?

On Sat, Jul 31, 2010 at 14:41, Christopher Nagel
 wrote:
> This may resolve to a fairly simple operation, but right now I'm a bit in
> the weeds and would like to ask your help in charting the best course
> forward.
> I administer the SVN repository for our workgroup.  I started it on a found
> copy of svn 1.3.9, but was fortunate to get the repository stored on a NAS
> mounted at /svnroot.  The SunOS box that svnserve runs on is being
> end-of-lifed and we've gotten a shiny new virtual linux box to use.  Sounds
> great, except now I have to make the move.
> I've considered the following:
> - add user svn to new system, ensure its UID/GIDs are same as on old system
> - install svn 1.3.9 on new system under new user
> - move the mount from old system to new, same mount point
> - restart svnserve and see if it works
> - upgrade svn to latest version in place
> Or:
> - create new svn user
> - install latest version of svn
> - export repository on old system (hopefully enough disk space is around)
> - wipe disk, mount storage on new system
> - create new repository
> - import old repository to new (history is preserved?)
> Or:
> - create new svn user and install latest svn on new system
> - move mount to new system
> - chown -R *
> - let svn update repository structure if it doesn't like it as-is
> What's the best way to do this?  I'm no linux admin, just a developer who
> likes safe code...

Your middle option is best, especially since you're changing OS here
too. Shut down svnserve (just to make sure no one commits after you've
started your migration - that won't be a deal-breaker but it adds some
extra work you'll have to do), then run svnadmin dump to get a
portable representation of the repository. Install the latest
Subversion on the Linux box, create the repository, then svnadmin load
to import your repository. As usual, the fine manual provides:
http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.html#svn.reposadmin.maint.migrate

By doing this, you'll keep your data safe *and* take advantage of the
many improvements made to Subversion and the back-end storage over the
years since 1.3 was released.

You'll also want to look over the Release Notes as they have some
information on upgrading as well.
http://subversion.apache.org/docs/release-notes/1.6.html
http://subversion.apache.org/docs/release-notes/1.5.html
http://subversion.apache.org/docs/release-notes/1.4.html

Re: Support for filesystem snapshots (?)


On 8/2/2010 3:46 PM, Vallon, Justin wrote:

From: Stefan Sperling [mailto:s...@elego.de]

  users asking interesting questions: 
http://mail-archives.apache.org/mod_mbox/subversion-users/201008.mbox/%3c6ec02a00cc9f684daf4af4084ca84d5f01c40...@drmbx3.winmail.deshaw.com%3e
  i dunno how fsfs behaves in face of an interrupted commit; whether or 
not it needs to be rolled back
  if you haven't touched current than the rev file will never be read 
and will be overwritten
  stsp: does that answer your question?
  i think so
  because the rev file of the following commit will have the same name to 
move things into place onto
  write lock only for revprop change and commit
  :-)

Now, how does rsync, or a file-system snapshot, know to make sure that
'current' is always copied first? Even if you copy 'current' first manually,
rsync might later overwrite it. But unless you use packing it's trivial to
fix the backup if it breaks, and all you risk is losing the most recent HEAD
revision, which you may not have gotten with a hotcopy anyway.


When I speak of a filesystem snapshot, I mean an instantaneous copy of the volume (ala 
NetApp, EMC, ZFS).  In this case, there is a guarantee that if we snap the new 
"current", then we will also have the other files (assuming that they have been 
flushed, etc, by the client).  Further, it sounds like (a) subsequent commits will not 
run into trouble because of the partial commit, and (b) the repository will not be 
otherwise affected by a partial commit.

That means filesystem snapshots pass the transactional test.


Maybe - is there a guarantee that the app flushes to disk in the 
expected order?  Or do snapshots take the current dirty filesystem 
buffers into account?


--
  Les Mikesell
   lesmikes...@gmail.com

RE: Support for filesystem snapshots (?)

From: Stefan Sperling [mailto:s...@elego.de] 
>  users asking interesting questions: 
> http://mail-archives.apache.org/mod_mbox/subversion-users/201008.mbox/%3c6ec02a00cc9f684daf4af4084ca84d5f01c40...@drmbx3.winmail.deshaw.com%3e
>  i dunno how fsfs behaves in face of an interrupted commit; whether or 
> not it needs to be rolled back
>  if you haven't touched current than the rev file will never be 
> read and will be overwritten
>  stsp: does that answer your question?
>  i think so
>  because the rev file of the following commit will have the same name 
> to move things into place onto
>  write lock only for revprop change and commit 
>  :-)
> 
> Now, how does rsync, or a file-system snapshot, know to make sure that
> 'current' is always copied first? Even if you copy 'current' first manually,
> rsync might later overwrite it. But unless you use packing it's trivial to
> fix the backup if it breaks, and all you risk is losing the most recent HEAD
> revision, which you may not have gotten with a hotcopy anyway.

When I speak of a filesystem snapshot, I mean an instantaneous copy of the 
volume (ala NetApp, EMC, ZFS).  In this case, there is a guarantee that if we 
snap the new "current", then we will also have the other files (assuming that 
they have been flushed, etc, by the client).  Further, it sounds like (a) 
subsequent commits will not run into trouble because of the partial commit, and 
(b) the repository will not be otherwise affected by a partial commit.

That means filesystem snapshots pass the transactional test.

rsync does not pass this transactional test due to order-of-copying.  rsync 
with some "current"-first logic would pass the test, but then we might as well 
use the hot-backup script.

--
-Justin

Re: Support for filesystem snapshots (?)

On Mon, Aug 02, 2010 at 03:25:48PM -0400, Vallon, Justin wrote:
> > E.g. Subversion's FSFS needs to create a revision file from the commit's
> > transaction, and move the finalized revision file into place.
> > After the revision file has been moved into place successfully, FSFS also
> > updates the svn:date revision property and moves the revision properties 
> > file
> > into place (or copies revprop data into an sqlite database if you use
> > revprop packing). Then, it updates the 'current' file which contains the
> > number of the current HEAD revision. If you use representation sharing to
> > save disk space, the commit may involve further updates to yet another
> > sqlite database.
> > 
> > All these actions need to complete in order to have a consistent state.
> >
> > If you're interested in seeing the code that does this, look at the
> > svn_fs_fs__commit() and commit_body() functions in
> > http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/fs_fs.c
> 
> I see this is executed with a FS write lock.  My concern would be
> focused on the interaction between the commit code and any rollback
> code.  For example, if the commit dies (any any point during the
> commit), what will be required to insure that the repository behaves
> as if the commit never started?  Will a repo cleanup be required; will
> the next committer cleanup the partial rev automatically (ie:
> overwrite stale files); will the repo be hopelessly inconsistent?

I honestly didn't know so I went and asked.
And learned something!

 users asking interesting questions: 
http://mail-archives.apache.org/mod_mbox/subversion-users/201008.mbox/%3c6ec02a00cc9f684daf4af4084ca84d5f01c40...@drmbx3.winmail.deshaw.com%3e
 i dunno how fsfs behaves in face of an interrupted commit; whether or 
not it needs to be rolled back
 if you haven't touched current than the rev file will never be read 
and will be overwritten
 stsp: does that answer your question?
 i think so
 because the rev file of the following commit will have the same name to 
move things into place onto
 write lock only for revprop change and commit 
 :-)
 so, using rsync for backup is fine?
 if you copy current first, yes
 what's hotcopy for then? just bdb?
 stsp: copying 'current'  first ... :-)

 ok, so what happens if I don't copy current first?
 you can copy revs/
 then a commit happens
 then you copy current
 so you don't have all of revs/ that current claims exist
 then I need to unwedge it
 by decrementing current
 right
 and hopefully you haven't just crossed a packing boundary
 eg if you want to decrement from 1002 to 999
 and someone packed it already
 a bit more work

So in the event that 'current' says you are at rN but the rev data in the
repository is still at r'N-1', the repository will complain (I've tried
that, "No such revision rN"), and you'll need to decrement the counter
in 'current'.  But otherwise, the repository will continue to work.

Now, how does rsync, or a file-system snapshot, know to make sure that
'current' is always copied first? Even if you copy 'current' first manually,
rsync might later overwrite it. But unless you use packing it's trivial to
fix the backup if it breaks, and all you risk is losing the most recent HEAD
revision, which you may not have gotten with a hotcopy anyway.

Still, I think I'll keep advising people to use hotcopy.
It avoids the problem with a too recent 'current' file, i.e. the backup
is always usable out of the box. And who knows how Subversion's on-disk
formats will at change in the future.
The hotcopy approach will always be supported, and works fine if, as you
pointed out, you can make sure that a hotcopy is being backup up while
not being written to.

Stefan

RE: Support for filesystem snapshots (?)

> From: Stefan Sperling [mailto:s...@elego.de] 
> On Mon, Aug 02, 2010 at 12:42:31PM -0400, Vallon, Justin wrote:
> > I did see that discussion, but it seems to contradict with the claim
> > that the database operations are transactional.
> 
> This is a frequent misunderstanding of the "atomic commit" concept.

I suppose that I am assuming that the repository operations would all be 
transactional.  Without transactional guarantees, you can have database 
corruption if a modification is interrupted.

> E.g. Subversion's FSFS needs to create a revision file from the commit's
> transaction, and move the finalized revision file into place.
> After the revision file has been moved into place successfully, FSFS also
> updates the svn:date revision property and moves the revision properties file
> into place (or copies revprop data into an sqlite database if you use
> revprop packing). Then, it updates the 'current' file which contains the
> number of the current HEAD revision. If you use representation sharing to
> save disk space, the commit may involve further updates to yet another
> sqlite database.
> 
> All these actions need to complete in order to have a consistent state.
>
> If you're interested in seeing the code that does this, look at the
> svn_fs_fs__commit() and commit_body() functions in
> http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/fs_fs.c

I see this is executed with a FS write lock.  My concern would be focused on 
the interaction between the commit code and any rollback code.  For example, if 
the commit dies (any any point during the commit), what will be required to 
insure that the repository behaves as if the commit never started?  Will a repo 
cleanup be required; will the next committer cleanup the partial rev 
automatically (ie: overwrite stale files); will the repo be hopelessly 
inconsistent?

A partially committed transaction still leaves the repository in a consistent 
state, so long as subsequent writers (or a special cleanup process) can undo 
the internal "damage", if any, made the partial commit.

> The safest way is creating a hotcopy of the live repository, and taking
> a copy of the hotcopy (by taking a snapshot of the filesystem the hotcopy
> resides in, or whatever). It's not the most efficient way of doing it,
> but the most portable and safest.

This solves the problem if you have control over both the hotcopy and the 
backup.  In a large environment, this means that the filesystem backup needs to 
be synchronized with the hotcopy, or else, the hotcopy itself might run 
*during* the backup, and would yield an inconsistent hotcopy 
(hotcopy-in-progress).

> You may be lucky and always get a consistent repository state by taking
> filesystem-level snapshots of a live repository, but there's no guarantee
> that the very latest commit will always be in a consistent state.

There will be two outcomes:

1) The in-progress transaction can always be rolled back (transparently or 
through the use of a recovery command), leaving an intact repository

2) The repository might be corrupted

If svn cannot guarantee outcome 1, then we are left with outcome 2, and that we 
can be left with a corrupt repository after a commit failure.

-- 
-Justin

RE: Support for filesystem snapshots (?)

From: Les Mikesell [mailto:lesmikes...@gmail.com] 
> On 8/2/2010 11:42 AM, Vallon, Justin wrote:
> > So, my follow up is:  If I unplug the network cable between svn-commiter 
> > and filesystem, will the repository be corrupt?
> >
> > If yes, then the underlying database is not atomic and therefore not 
> > transactional.
> >
> > If no, then the server can take a snapshot at any point in time, and the 
> > snapshot is guaranteed to be consistent.
>
> A better match for comparison would be what happens if the server 
> crashes mid-transaction.  A backup taken with the server active might 
> catch the same state.  So the more relevant question would be if the 
> repository will always be consistent after a server crash.

That is the situation I raised.  If the network connection between the host 
that is modifying the repository and the filesystem that houses the repository 
is lost, will be repository be (a) corrupt, (b) require cleanup (locks, etc), 
(c) pristine?

(c) is great
(b) is ok
(a) means the operations are not transactional

Further:

(c) snapshots work
(b) snapshots work, but need a cleanup after restore
(a) snapshots don't work; repository is not transactional, either

-- 
-Justin

Re: Unclear syntax for relative addressing of svn:externals, on RHEL 5, subversion-1.6.12

On Tue, Jul 13, 2010 at 09:08:27AM -0700, Blair Zajac wrote:
> On 07/13/2010 05:15 AM, Stefan Sperling wrote:
> >I think we should discourage use of the old syntax in both the output
> >of svn help and in the book. Diff to the help text below.
> >Would this improve things?
> 
> Sounds good to me.
> 
> I think this last sentence starting with "The ambiguous format..."
> should go before 1.4 is mentioned because it is relevant to the 1.5
> format.

Good catch, thanks.
Committed with further tweaks and some typos fixed in r981653.
Sorry about the delay.

Does anyone feel like submitting a similar patch to the book?
In particular about making the book discourage use of the 1.4-and-earlier
syntax?

Thanks,
Stefan

Re: Support for filesystem snapshots (?)

On Mon, Aug 02, 2010 at 12:42:31PM -0400, Vallon, Justin wrote:
> I did see that discussion, but it seems to contradict with the claim
> that the database operations are transactional.

This is a frequent misunderstanding of the "atomic commit" concept.

> So, my follow up is:  If I unplug the network cable between
> svn-commiter and filesystem, will the repository be corrupt?

When you break the network connection, commits either pass or fail.
What can happen is that you are left with stale transactions lying
around. That's all. The critical section comes later, when the commit
is to be finalized. At that point, all data has been transmitted across
the network already.

> If yes, then the underlying database is not atomic and therefore not
> transactional.

No, that's not the case. Just because unplugging the network cable
does not cause havoc does not mean the underlying filesystem is
truly transactional at the lowest level.

Commits are transactional from the user's point of view.
Or you could say, they're transactional from the point of view of
every layer in Subversion *above* the filesystem layer.

If two people commit to the same path at the same time, one commit
will succeed and the other will fail. That's what atomic commits
are about. They're not about consistency of the repository filesystem.

E.g. Subversion's FSFS needs to create a revision file from the commit's
transaction, and move the finalized revision file into place.
After the revision file has been moved into place successfully, FSFS also
updates the svn:date revision property and moves the revision properties file
into place (or copies revprop data into an sqlite database if you use
revprop packing). Then, it updates the 'current' file which contains the
number of the current HEAD revision. If you use representation sharing to
save disk space, the commit may involve further updates to yet another
sqlite database.

All these actions need to complete in order to have a consistent state.

If you're interested in seeing the code that does this, look at the
svn_fs_fs__commit() and commit_body() functions in
http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/fs_fs.c

> If no, then the server can take a snapshot at any point in time, and
> the snapshot is guaranteed to be consistent.

The safest way is creating a hotcopy of the live repository, and taking
a copy of the hotcopy (by taking a snapshot of the filesystem the hotcopy
resides in, or whatever). It's not the most efficient way of doing it,
but the most portable and safest.

You may be lucky and always get a consistent repository state by taking
filesystem-level snapshots of a live repository, but there's no guarantee
that the very latest commit will always be in a consistent state.

Stefan

Re: Support for filesystem snapshots (?)


On 8/2/2010 11:42 AM, Vallon, Justin wrote:

I did see that discussion, but it seems to contradict with the claim that the 
database operations are transactional.

So, my follow up is:  If I unplug the network cable between svn-commiter and 
filesystem, will the repository be corrupt?

If yes, then the underlying database is not atomic and therefore not 
transactional.

If no, then the server can take a snapshot at any point in time, and the 
snapshot is guaranteed to be consistent.


That's not a good match for the operations at least in server-mode 
operations since the server can notice the disconnect and back out 
failed transactions.  And even with file-mapped connections, the 
committing client should know that the commit did not complete.


A better match for comparison would be what happens if the server 
crashes mid-transaction.  A backup taken with the server active might 
catch the same state.  So the more relevant question would be if the 
repository will always be consistent after a server crash.


--
  Les Mikesell
   lesmikes...@gmail.com

RE: Support for filesystem snapshots (?)

I did see that discussion, but it seems to contradict with the claim that the 
database operations are transactional.

So, my follow up is:  If I unplug the network cable between svn-commiter and 
filesystem, will the repository be corrupt?

If yes, then the underlying database is not atomic and therefore not 
transactional.

If no, then the server can take a snapshot at any point in time, and the 
snapshot is guaranteed to be consistent.

-- 
-Justin


-Original Message-
From: Les Mikesell [mailto:lesmikes...@gmail.com] 
Sent: Monday, August 02, 2010 12:30 PM
To: users@subversion.apache.org
Subject: Re: Support for filesystem snapshots (?)

On 8/2/2010 11:17 AM, Stefan Sperling wrote:
> On Mon, Aug 02, 2010 at 11:38:30AM -0400, Vallon, Justin wrote:
>> In the svn book, 
>> http://svnbook.red-bean.com/en/1.1/ch05s03.html#svn-ch-5-sect-3.6 on 
>> "Repository Backup" describes some backup methods - hot copy, etc.  Given 
>> that enterprise-level filesystems generally support filesystem snapshots, 
>> what is SVN's position on whether such snapshots are sufficient for backups?
>>
>> Suppose our criteria is that we unplug the network cable between the writer 
>> (svn commit) and the filesystem.  If unplugging the network cable could 
>> yield a repository that is corrupt, then doesn't that mean that there are 
>> failure modes where the repository is corrupted?  If a filesystem insures 
>> that the snapshots behave in this way (as if you disconnect the network 
>> cable, make an instant copy, then reconnect), it would seem to imply that 
>> these volume-level snapshots would result in a consistent view.  If the 
>> snapshots are not consistent, then atomicity is not being insured (somewhere 
>> between application, client o/s, server o/s, disk controller, media).
>>
>> My follow up is whether the svn book could include a word about whether it 
>> blesses (or not) this form of backup?
>
> By chance, this very topic was discussed earlier today.
> See http://svn.haxx.se/users/archive-2010-08/0019.shtml
>
> Quote:
> "Use 'svnadmin hotcopy' to create a copy of a repository that is consistent
> from Subversion's point of view. There is no other repository copying
> tool that can guarantee a consistent repository snapshot."
>
> This holds true for filesystem-level snapshots.
> The atomicity of Subversion's commits only exists from the application's
> point of view.

Are there any svnadmin commands that could quiesce/lock the repository 
in a consistent state for the duration of a filesystem snapshot - or if 
not, could that be added as an enhancement?

-- 
   Les Mikesell
lesmikes...@gmail.com

Re: Support for filesystem snapshots (?)

On Mon, Aug 02, 2010 at 11:29:44AM -0500, Les Mikesell wrote:
> On 8/2/2010 11:17 AM, Stefan Sperling wrote:
> >The atomicity of Subversion's commits only exists from the application's
> >point of view.
> 
> Are there any svnadmin commands that could quiesce/lock the
> repository in a consistent state for the duration of a filesystem
> snapshot - or if not, could that be added as an enhancement?

None that I'm aware of.

I guess we could add a command to svnadmin that obtains a read lock
(blocking all writers) and waits forever until it is killed.
If that's considered useful.

However, this may cause all repository snapshots to be read-locked as well
once they're restored from backup.
So a mechanism to detect stale locks might also need to be devised.
I'm not sure if such a mechanism is already present.

Stefan

RE: SVN "Relay"

As far as I can tell, from the "vulnerability":

<<<
GENERATES A FULL SSL CERTIFICATE WHICH THE VICTIM'S BROWSER
WILL PROMPT HIM TO ACCEPT:

webmitm -dd
>>>

The user needs to accept the man-in-the-middle certificate.

-- 
-Justin

-Original Message-
From: Istace Emmanuel [mailto:istace.emman...@hotmail.com] 
Sent: Monday, August 02, 2010 12:01 PM
To: 'Les Mikesell'
Cc: users@subversion.apache.org
Subject: RE: SVN "Relay"

" Can you point me to something specific?"
No problem, here a video (the "fun" side) :
http://www.youtube.com/watch?v=Aak6-B3JORE
An article :
http://forums.remote-exploit.org/tutorials-guides/19852-ssl-spoof-using-wire
shark-decode-ssl-packets.html

"If you are concerned about your service provider maybe you should use
someone else - or a service that lets you run your own system images where
you could set up a blowfish-based vpn."
I haven't choose that :(
But as i say, it's a temporary solution

-Message d'origine-
De : Les Mikesell [mailto:lesmikes...@gmail.com] 
Envoyé : lundi 2 août 2010 17:07
À : users@subversion.apache.org
Objet : Re: SVN "Relay"

On 8/2/2010 8:56 AM, Istace Emmanuel wrote:
> " Should I be worried about banking transactions or credit card orders?"
> Yeah :(
>
> " You could use any kind of VPN you want with the remote site.  Use an 
> IPSEC tunnel between hosts if you don't trust SSL.  Or OpenVPN with
blowfish."
> No, because the SVN is on a SaaS cloud, so we just have access to the 
> service and not the system. So we can't install a VPN server and 
> remember, vpn and ipsec use SSL. Search on google about SSL Spoofing 
> ;)

Can you point me to something specific?  I see things about spoofing some
other site's certificate and some things about specific implementations
being subject to man-in-the-middle attacks but nothing that looks like a
generic weakness.  If you are concerned about your service provider (who
would have the best opportunity to arrange a man-in-the-middle connection),
maybe you should use someone else - or a service that lets you run your own
system images where you could set up a blowfish-based vpn.

--
  Les Mikesell
lesmikes...@gmail.com

Re: Support for filesystem snapshots (?)


On 8/2/2010 11:17 AM, Stefan Sperling wrote:

On Mon, Aug 02, 2010 at 11:38:30AM -0400, Vallon, Justin wrote:

In the svn book, http://svnbook.red-bean.com/en/1.1/ch05s03.html#svn-ch-5-sect-3.6 on 
"Repository Backup" describes some backup methods - hot copy, etc.  Given that 
enterprise-level filesystems generally support filesystem snapshots, what is SVN's 
position on whether such snapshots are sufficient for backups?

Suppose our criteria is that we unplug the network cable between the writer 
(svn commit) and the filesystem.  If unplugging the network cable could yield a 
repository that is corrupt, then doesn't that mean that there are failure modes 
where the repository is corrupted?  If a filesystem insures that the snapshots 
behave in this way (as if you disconnect the network cable, make an instant 
copy, then reconnect), it would seem to imply that these volume-level snapshots 
would result in a consistent view.  If the snapshots are not consistent, then 
atomicity is not being insured (somewhere between application, client o/s, 
server o/s, disk controller, media).

My follow up is whether the svn book could include a word about whether it 
blesses (or not) this form of backup?


By chance, this very topic was discussed earlier today.
See http://svn.haxx.se/users/archive-2010-08/0019.shtml

Quote:
"Use 'svnadmin hotcopy' to create a copy of a repository that is consistent
from Subversion's point of view. There is no other repository copying
tool that can guarantee a consistent repository snapshot."

This holds true for filesystem-level snapshots.
The atomicity of Subversion's commits only exists from the application's
point of view.


Are there any svnadmin commands that could quiesce/lock the repository 
in a consistent state for the duration of a filesystem snapshot - or if 
not, could that be added as an enhancement?


--
  Les Mikesell
   lesmikes...@gmail.com

Re: Support for filesystem snapshots (?)

On Mon, Aug 02, 2010 at 11:38:30AM -0400, Vallon, Justin wrote:
> In the svn book, 
> http://svnbook.red-bean.com/en/1.1/ch05s03.html#svn-ch-5-sect-3.6 on 
> "Repository Backup" describes some backup methods - hot copy, etc.  Given 
> that enterprise-level filesystems generally support filesystem snapshots, 
> what is SVN's position on whether such snapshots are sufficient for backups?
> 
> Suppose our criteria is that we unplug the network cable between the writer 
> (svn commit) and the filesystem.  If unplugging the network cable could yield 
> a repository that is corrupt, then doesn't that mean that there are failure 
> modes where the repository is corrupted?  If a filesystem insures that the 
> snapshots behave in this way (as if you disconnect the network cable, make an 
> instant copy, then reconnect), it would seem to imply that these volume-level 
> snapshots would result in a consistent view.  If the snapshots are not 
> consistent, then atomicity is not being insured (somewhere between 
> application, client o/s, server o/s, disk controller, media).
> 
> My follow up is whether the svn book could include a word about whether it 
> blesses (or not) this form of backup?

By chance, this very topic was discussed earlier today.
See http://svn.haxx.se/users/archive-2010-08/0019.shtml

Quote:
"Use 'svnadmin hotcopy' to create a copy of a repository that is consistent
from Subversion's point of view. There is no other repository copying
tool that can guarantee a consistent repository snapshot."

This holds true for filesystem-level snapshots.
The atomicity of Subversion's commits only exists from the application's
point of view.

Stefan

RE: SVN "Relay"

" Can you point me to something specific?"
No problem, here a video (the "fun" side) :
http://www.youtube.com/watch?v=Aak6-B3JORE
An article :
http://forums.remote-exploit.org/tutorials-guides/19852-ssl-spoof-using-wire
shark-decode-ssl-packets.html

"If you are concerned about your service provider maybe you should use
someone else - or a service that lets you run your own system images where
you could set up a blowfish-based vpn."
I haven't choose that :(
But as i say, it's a temporary solution

-Message d'origine-
De : Les Mikesell [mailto:lesmikes...@gmail.com] 
Envoyé : lundi 2 août 2010 17:07
À : users@subversion.apache.org
Objet : Re: SVN "Relay"

On 8/2/2010 8:56 AM, Istace Emmanuel wrote:
> " Should I be worried about banking transactions or credit card orders?"
> Yeah :(
>
> " You could use any kind of VPN you want with the remote site.  Use an 
> IPSEC tunnel between hosts if you don't trust SSL.  Or OpenVPN with
blowfish."
> No, because the SVN is on a SaaS cloud, so we just have access to the 
> service and not the system. So we can't install a VPN server and 
> remember, vpn and ipsec use SSL. Search on google about SSL Spoofing 
> ;)

Can you point me to something specific?  I see things about spoofing some
other site's certificate and some things about specific implementations
being subject to man-in-the-middle attacks but nothing that looks like a
generic weakness.  If you are concerned about your service provider (who
would have the best opportunity to arrange a man-in-the-middle connection),
maybe you should use someone else - or a service that lets you run your own
system images where you could set up a blowfish-based vpn.

--
  Les Mikesell
lesmikes...@gmail.com

Support for filesystem snapshots (?)

In the svn book, 
http://svnbook.red-bean.com/en/1.1/ch05s03.html#svn-ch-5-sect-3.6 on 
"Repository Backup" describes some backup methods - hot copy, etc.  Given that 
enterprise-level filesystems generally support filesystem snapshots, what is 
SVN's position on whether such snapshots are sufficient for backups?

Suppose our criteria is that we unplug the network cable between the writer 
(svn commit) and the filesystem.  If unplugging the network cable could yield a 
repository that is corrupt, then doesn't that mean that there are failure modes 
where the repository is corrupted?  If a filesystem insures that the snapshots 
behave in this way (as if you disconnect the network cable, make an instant 
copy, then reconnect), it would seem to imply that these volume-level snapshots 
would result in a consistent view.  If the snapshots are not consistent, then 
atomicity is not being insured (somewhere between application, client o/s, 
server o/s, disk controller, media).

My follow up is whether the svn book could include a word about whether it 
blesses (or not) this form of backup?

--
-Justin

Re: SVN "Relay"


On 8/2/2010 8:56 AM, Istace Emmanuel wrote:

" Should I be worried about banking transactions or credit card orders?"
Yeah :(

" You could use any kind of VPN you want with the remote site.  Use an IPSEC
tunnel between hosts if you don't trust SSL.  Or OpenVPN with blowfish."
No, because the SVN is on a SaaS cloud, so we just have access to the
service and not the system. So we can't install a VPN server and remember,
vpn and ipsec use SSL. Search on google about SSL Spoofing ;)


Can you point me to something specific?  I see things about spoofing 
some other site's certificate and some things about specific 
implementations being subject to man-in-the-middle attacks but nothing 
that looks like a generic weakness.  If you are concerned about your 
service provider (who would have the best opportunity to arrange a 
man-in-the-middle connection), maybe you should use someone else - or a 
service that lets you run your own system images where you could set up 
a blowfish-based vpn.


--
 Les Mikesell
   lesmikes...@gmail.com

Re: SVN "Relay"

2010-08-02 Thread 'Stefan Sperling'

On Mon, Aug 02, 2010 at 01:40:38PM +0100, Giulio Troccoli wrote:
> 
> > And if you decide to use svnsync for mirroring, note that the
> > way the book describes the mirroring setup is vulnerable to a
> > race condition.
> > Make sure to read this issue:
> > http://subversion.tigris.org/issues/show_bug.cgi?id=3546
> > To avoid the problem described in issue #3546, you should
> > make the mirror repository pull changes from the master via a
> > cronjob, and synchronise svnsync jobs using a file lock on
> > the computer hosting the mirror repository.
> > Only one svnsync job must be writing to the mirror repository
> > at a time, or you risk breaking sync jobs. So do not push
> > revisions from the master to the mirror via the post-commit
> > hook (which is what the book suggests).
> > Something like the following should work from a crontab on
> > Linux systems:
> > 0 * * * * flock -w 60 /tmp/svnsync.lock svnsync sync
> > file:///path/to/repository/mirror
> >
> > We're working on fixing this problem such that the way the
> > book describes it will work without a race condition, but
> > that work is not yet complete.
> 
> A bit OT, I just want to check that this (the race condition) does not
> apply to two svnsync processes trying to sync two different
> repositories.

Unless both svnsync processes are writing to the same repository,
the race condition cannot trigger.

> I use svnsync to keep our DR server up-to-date. We have very few
> people committing to the various repositories so it's more likely (and
> still it's not that likely) that two people commit to two different
> repositories than to the same one, so I just wanted to check thise
> scenario with you.

It has nothing much to do with when commits happen.

The problem is that two svnsync processes running simultaneously
(for whatever reason) can corrupt svnsync meta-data, causing subsequent
syncs to fail because the actual latest revision in the repository doesn't
match what svnsync expects.

See here for a more detailed description of the problem:
http://www.mail-archive.com/d...@subversion.apache.org/msg00265.html

Stefan

RE: SVN "Relay"

" Should I be worried about banking transactions or credit card orders?"
Yeah :(

" You could use any kind of VPN you want with the remote site.  Use an IPSEC
tunnel between hosts if you don't trust SSL.  Or OpenVPN with blowfish."
No, because the SVN is on a SaaS cloud, so we just have access to the
service and not the system. So we can't install a VPN server and remember,
vpn and ipsec use SSL. Search on google about SSL Spoofing ;)

To Tennebø Frode : Ok, i read this tonight, thank's for your help to all, i
think i have what i want and all the docs.

Re: Moving NAS owner and upgrading?

2010-08-02 Thread Andy Levy

On Sat, Jul 31, 2010 at 14:41, Christopher Nagel
 wrote:
> This may resolve to a fairly simple operation, but right now I'm a bit in
> the weeds and would like to ask your help in charting the best course
> forward.
> I administer the SVN repository for our workgroup.  I started it on a found
> copy of svn 1.3.9, but was fortunate to get the repository stored on a NAS
> mounted at /svnroot.  The SunOS box that svnserve runs on is being
> end-of-lifed and we've gotten a shiny new virtual linux box to use.  Sounds
> great, except now I have to make the move.
> I've considered the following:
> - add user svn to new system, ensure its UID/GIDs are same as on old system
> - install svn 1.3.9 on new system under new user
> - move the mount from old system to new, same mount point
> - restart svnserve and see if it works
> - upgrade svn to latest version in place
> Or:
> - create new svn user
> - install latest version of svn
> - export repository on old system (hopefully enough disk space is around)
> - wipe disk, mount storage on new system
> - create new repository
> - import old repository to new (history is preserved?)
> Or:
> - create new svn user and install latest svn on new system
> - move mount to new system
> - chown -R *
> - let svn update repository structure if it doesn't like it as-is
> What's the best way to do this?  I'm no linux admin, just a developer who
> likes safe code...

Your middle option is best, especially since you're changing OS here
too. Shut down svnserve (just to make sure no one commits after you've
started your migration - that won't be a deal-breaker but it adds some
extra work you'll have to do), then run svnadmin dump to get a
portable representation of the repository. Install the latest
Subversion on the Linux box, create the repository, then svnadmin load
to import your repository. As usual, the fine manual provides:
http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.html#svn.reposadmin.maint.migrate

By doing this, you'll keep your data safe *and* take advantage of the
many improvements made to Subversion and the back-end storage over the
years since 1.3 was released.

You'll also want to look over the Release Notes as they have some
information on upgrading as well.
http://subversion.apache.org/docs/release-notes/1.6.html
http://subversion.apache.org/docs/release-notes/1.5.html
http://subversion.apache.org/docs/release-notes/1.4.html

Re: Path-Based Authorization for Tags

2010-08-02 Thread Johan Corveleyn

On Wed, Jul 28, 2010 at 7:59 PM, Luiz Guilherme Kimel  wrote:
> Hello there,
>
>
>
> I’m implementing a subversion repository that keeps project management
> artefacts. In other words, we are using SVN for document management. And
> everything was going right until we find that managers need to keep status
> of allocated human resources salaries and this can’t be accessed by everyone
> in the project. Ok, let’s use path-based auth, but then comes a question:
> what to do with tags?
>
>
>
> Example path-based settings for a project:
>
>
>
> [svn:/projects/project-abc/tags/]
>
> @team=r
>
> @cm=rw
>
> [svn:/projects/project-abc/trunk/]
>
> @team=rw
>
> [svn:/projects/project-abc/trunk/releases]
>
> @team=r
>
> @cm=rw
>
> [svn:/projects/project-abc/trunk/planning]
>
> # no change
>
> [svn:/projects/project-abc/trunk/costs]
>
> @team=
>
> @maganement=rw
>
>
>
> My requirements:
>
>
>
> 1)  I want that the “costs” directory will be writeable only by
> management group while under the trunk folder;
>
> 2)  I want that the “costs” directory will be readeable and just
> readeable to management group once under tags folder;
>
> 3)  I don’t want to have to specify different path-based auth rules for
> each new tag…
>
> 4)  It’s desireable that the configuration management group (cm group)
> is able to generate tags, but still can’t read the “costs” directory!
>
>
>
> Am I dreaming? It would be wonderful if I were able to specify path-based
> auth like this:
>
>
>
> [svn:/projects/project-abc/tags/*/costs]
>
> *=
>
> @maganement=r
>
>
>
> [svn:/projects/project-abc/trunk/costs]
>
> *=
>
> @maganement=rw
>

Unfortunately, you can't do that right now.

FYI, there is an open enhancement request in the issue tracker for
this feature:
http://subversion.tigris.org/issues/show_bug.cgi?id=2662 - authz with wildcards

It seems to be a very useful feature, which comes up quite frequently
on this list. Some patches have been proposed in the issue tracker,
but for some reason they haven't been applied...

-- 
Johan

Moving NAS owner and upgrading?

2010-08-02 Thread Christopher Nagel

This may resolve to a fairly simple operation, but right now I'm a bit in the 
weeds and would like to ask your help in charting the best course forward.

I administer the SVN repository for our workgroup.  I started it on a found 
copy 
of svn 1.3.9, but was fortunate to get the repository stored on a NAS mounted 
at 
/svnroot.  The SunOS box that svnserve runs on is being end-of-lifed and we've 
gotten a shiny new virtual linux box to use.  Sounds great, except now I have 
to 
make the move.

I've considered the following:
- add user svn to new system, ensure its UID/GIDs are same as on old system
- install svn 1.3.9 on new system under new user
- move the mount from old system to new, same mount point
- restart svnserve and see if it works
- upgrade svn to latest version in place

Or:
- create new svn user
- install latest version of svn
- export repository on old system (hopefully enough disk space is around)
- wipe disk, mount storage on new system
- create new repository
- import old repository to new (history is preserved?)

Or: 
- create new svn user and install latest svn on new system
- move mount to new system
- chown -R *
- let svn update repository structure if it doesn't like it as-is

What's the best way to do this?  I'm no linux admin, just a developer who likes 
safe code...

Thanks!
Chris

Re: svnserv + ssh + ldap

2010-08-02 Thread Mark Phippard

On Mon, Aug 2, 2010 at 8:22 AM, Nico Kadel-Garcia  wrote:
> Good, but ouch. Let's try adding a bit of rigor, shall we? First,
> before running such a daemon, always check that it actually exists,
> where you expect it to exist. Running random commands that will handle
> passwords which may have been replaced by who knows what somehwere in
> your $PATH is always a bad idea. So let's use this instead:

This is a conversation, not a user guide.  I was just giving some
basic example of how you can use it without X.

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

Re: SVN "Relay"


Istace Emmanuel wrote:


Ok, but that's not what i we at work. The local svn will be a "working copy"
of the external svn. In fact, no user have to use the external svn if
there's a local svn. The goal of this is the limit the access to the WAN. If
the external svn is sync twice per day, we can limit these access and
decrease the commit time for internal user (we have no external user).


If you have no external users, why not put the main repository on the LAN 
instead of a copy?  If you need an offsite copy for backup, do the svnsync 
periodically to the WAN copy where you don't have to worry about any other commits.



A LAN
commit is faster than a WAN commit (the bandwith is lower for wan) And we
can have a better security, we have no control on what's do on the WAN with
our data and many commit from LAN to WAN are "point of access to private
data" (SSL or not, ssl is not a good security, you can spoof him easily).


Should I be worried about banking transactions or credit card orders?


We
can limit this risk by limitting the data who are sent. 



You could use any kind of VPN you want with the remote site.  Use an IPSEC 
tunnel between hosts if you don't trust SSL.  Or OpenVPN with blowfish.


--
  Les Mikesell
   lesmikes...@gmail.com

RE: SVN "Relay"

2010-08-02 Thread Giulio Troccoli


> And if you decide to use svnsync for mirroring, note that the
> way the book describes the mirroring setup is vulnerable to a
> race condition.
> Make sure to read this issue:
> http://subversion.tigris.org/issues/show_bug.cgi?id=3546
> To avoid the problem described in issue #3546, you should
> make the mirror repository pull changes from the master via a
> cronjob, and synchronise svnsync jobs using a file lock on
> the computer hosting the mirror repository.
> Only one svnsync job must be writing to the mirror repository
> at a time, or you risk breaking sync jobs. So do not push
> revisions from the master to the mirror via the post-commit
> hook (which is what the book suggests).
> Something like the following should work from a crontab on
> Linux systems:
> 0 * * * * flock -w 60 /tmp/svnsync.lock svnsync sync
> file:///path/to/repository/mirror
>
> We're working on fixing this problem such that the way the
> book describes it will work without a race condition, but
> that work is not yet complete.

A bit OT, I just want to check that this (the race condition) does not apply to 
two svnsync processes trying to sync two different repositories.

I use svnsync to keep our DR server up-to-date. We have very few people 
committing to the various repositories so it's more likely (and still it's not 
that likely) that two people commit to two different repositories than to the 
same one, so I just wanted to check thise scenario with you.

Giulio


Linedata Limited
Registered Office: 85 Gracechurch St., London, EC3V 0AA
Registered in England and Wales No 3475006 VAT Reg No 710 3140 03

Re: SVN "Relay"

On Mon, Aug 02, 2010 at 07:50:49AM -0400, Nico Kadel-Garcia wrote:
> On Mon, Aug 2, 2010 at 7:41 AM, Tennebø Frode
>  wrote:
> >> Ok, but that's not what i we at work. The local svn will be a
> >> "working copy"
> >> of the external svn. In fact, no user have to use the
> >> external svn if there's a local svn. The goal of this is the
> >> limit the access to the WAN.
> >
> > In that case the WAN repository can be looked on as a mirror/off-line 
> > backup of the LAN repository.
> >
> > You could use rsync to keep an up-to-date mirror on the WAN while the LAN 
> > is the active server.  You need to set up an svn installation on the LAN, 
> > rsync (or some other form of copy) the initial repository from the WAN 
> > server to the LAN server.  All uses now use the LAN repository.  Some 
> > cron-job automatically rsyncs the LAN repository back to the WAN server.
> 
> Only if the rsync is done as part of a post-commit operation:
> otherwise, what are "atomic" operations for the Subversion repository
> may be duplicated when only partially complete, and chaos may ensue.

No. That's bad advice. Running rsync from the post-commit hook won't
magically make the copy operation atomic.

rsync will never be able to create a consistent snapshot of a live
Subversion repository. Use rsync only on repositories that are known
to be consistent and that no commits or revprop edits are being made to.

Use 'svnadmin hotcopy' to create a copy of a repository that is consistent
from Subversion's point of view. There is no other repository copying
tool that can guarantee a consistent repository snapshot.
Then mirror the hotcopy with any tool of your liking.

Another way of doing it is dump/load. That will also be atomic,
but won't copy hooks and configs, only stuff beneath the db/ directory.
Making a hotcopy is much faster than creating a dump of the entire repository.

Additionally, creating incremental dumps of revisions from within the
post-commit hook is a good way of getting a backup of each revision in
virtually real time. In case of desaster, you can use these partial dumps
to fill up your backup repository to the last committed revision (or at
least very close to there). Note that a fresh full dump/hotcopy is required
every now and then if you allow people to change revprops (e.g. svn:log).

You can also load the incremental dumps of each revisions on your
repository mirror. This achieves the same effect as svnsync.

Istace, just to make sure you've understood this: You need to have
one repository which people commit to. You can create a mirror repository,
but the mirror should only be used for read-only operations.
You cannot write to the mirror and expect the changes reach the
master repository. Subversion doesn't work that way.
See also this section of the book:
http://svnbook.red-bean.com/nightly/en/svn.serverconfig.httpd.html#svn.serverconfig.httpd.extra.writethruproxy

And if you decide to use svnsync for mirroring, note that the way the
book describes the mirroring setup is vulnerable to a race condition.
Make sure to read this issue:
http://subversion.tigris.org/issues/show_bug.cgi?id=3546
To avoid the problem described in issue #3546, you should make the mirror
repository pull changes from the master via a cronjob, and synchronise
svnsync jobs using a file lock on the computer hosting the mirror repository.
Only one svnsync job must be writing to the mirror repository at a time,
or you risk breaking sync jobs. So do not push revisions from the master
to the mirror via the post-commit hook (which is what the book suggests).
Something like the following should work from a crontab on Linux systems:
0 * * * * flock -w 60 /tmp/svnsync.lock svnsync sync 
file:///path/to/repository/mirror

We're working on fixing this problem such that the way the book
describes it will work without a race condition, but that work is not
yet complete.

Stefan

Re: svnserv + ssh + ldap

2010-08-02 Thread Nico Kadel-Garcia

On Sun, Aug 1, 2010 at 1:37 PM, Mark Phippard  wrote:
> On Sun, Aug 1, 2010 at 12:59 PM, Nico Kadel-Garcia  wrote:
>>> AFAIK it's possible to run gnome-keyring without X.
>>
>> It's painful. Take a glance at
>> http://superuser.com/questions/141036/use-of-gnome-keyring-daemon-without-x,
>> which documents manually editing /etc/pam.d/ login settings. It's also
>> heavily dependent on X libraries.
>
> Subversion and GNOME keyring work well without X.  In your login
> script you just want:
>
> $ export `gnome-keyring-daemon`

Good, but ouch. Let's try adding a bit of rigor, shall we? First,
before running such a daemon, always check that it actually exists,
where you expect it to exist. Running random commands that will handle
passwords which may have been replaced by who knows what somehwere in
your $PATH is always a bad idea. So let's use this instead:

if [ -x /usr/bin/gnome-keyring-daemon ]; then
export `gnome-keyring-daemon`
fi

And .login is not universal. .bash_profile, for example, is more
effective for most bash users. But I take the idea.

Last, it doesn't store the settings for your existing keyrings
anywhere so you don't wind up creating one every time you log in. This
sort of setting is stored for SSH keys by tools like the Perl script
'keychain': I'm not sure there's one like it for the
gnome-keyring-daemon. So it's reasonable, but resource consumptive if
you run multiple logins to the same host, and will tend to clutter
/tmp/. That's serviceable, so I'll actually try that out this week.

> Subversion will prompt you for password to unlock the keyring when it
> is locked.  In logout script you want:
>
> $ kill $GNOME_KEYRING_PID

Nothing personal, but you seem to be casually slapping things into
your configuration scripts with uncertain provenance. The resulting
errors are begging to cause issues in multiplatform environments. Try
something like this instead

 [ -n "$GNOME_KEYRING_PID" ] && kill $GNOME_KEYRING_PID

And for bash users, it goes in .bash_logout.

> One problem is that GNOME does not provide any CLI for working with
> the keyring.  CollabNet created a keyring_tool that lets you do this.
> It is packaged in the RPM we provide for Linux.

Good! Is there some reason it's not in the main Subversion codeline?
Or in a forked off product?

Re: SVN "Relay"

2010-08-02 Thread Nico Kadel-Garcia

On Mon, Aug 2, 2010 at 7:41 AM, Tennebø Frode
 wrote:
>> Ok, but that's not what i we at work. The local svn will be a
>> "working copy"
>> of the external svn. In fact, no user have to use the
>> external svn if there's a local svn. The goal of this is the
>> limit the access to the WAN.
>
> In that case the WAN repository can be looked on as a mirror/off-line backup 
> of the LAN repository.
>
> You could use rsync to keep an up-to-date mirror on the WAN while the LAN is 
> the active server.  You need to set up an svn installation on the LAN, rsync 
> (or some other form of copy) the initial repository from the WAN server to 
> the LAN server.  All uses now use the LAN repository.  Some cron-job 
> automatically rsyncs the LAN repository back to the WAN server.

Only if the rsync is done as part of a post-commit operation:
otherwise, what are "atomic" operations for the Subversion repository
may be duplicated when only partially complete, and chaos may ensue.

RE: SVN "Relay"

2010-08-02 Thread Tennebø Frode

> Ok, but that's not what i we at work. The local svn will be a 
> "working copy"
> of the external svn. In fact, no user have to use the 
> external svn if there's a local svn. The goal of this is the 
> limit the access to the WAN.

In that case the WAN repository can be looked on as a mirror/off-line backup of 
the LAN repository.

You could use rsync to keep an up-to-date mirror on the WAN while the LAN is 
the active server.  You need to set up an svn installation on the LAN, rsync 
(or some other form of copy) the initial repository from the WAN server to the 
LAN server.  All uses now use the LAN repository.  Some cron-job automatically 
rsyncs the LAN repository back to the WAN server.

Make sure that only this mechanism can touch the WAN repository or you are 
FUBARed.

 -Frode
-- 
^ Frode Tennebø   | email: frode.tenn...@saabgroup.com ^
| Saab Technologies Norway AS | Isebakkeveien 49   |
| N-1788 Halden   | Phone: +47 45 24 99 39 |
| with Standard.Disclaimer; use Standard.Disclaimer;   |

Re: SVN "Relay"

2010-08-02 Thread Ryan Schmidt

On Aug 2, 2010, at 05:16, Istace Emmanuel wrote:

> Ok, but that's not what i we at work. The local svn will be a "working copy"
> of the external svn. In fact, no user have to use the external svn if
> there's a local svn. The goal of this is the limit the access to the WAN. If
> the external svn is sync twice per day, we can limit these access and
> decrease the commit time for internal user (we have no external user). A LAN
> commit is faster than a WAN commit (the bandwith is lower for wan) And we
> can have a better security, we have no control on what's do on the WAN with
> our data and many commit from LAN to WAN are "point of access to private
> data" (SSL or not, ssl is not a good security, you can spoof him easily). We
> can limit this risk by limitting the data who are sent. 

Ok, then I don't think svnsync will help. That's not how it works (and I don't 
know of anything that does work that way for Subversion, but I haven't done an 
exhaustive search).

RE: SVN "Relay"

Ok, but that's not what i we at work. The local svn will be a "working copy"
of the external svn. In fact, no user have to use the external svn if
there's a local svn. The goal of this is the limit the access to the WAN. If
the external svn is sync twice per day, we can limit these access and
decrease the commit time for internal user (we have no external user). A LAN
commit is faster than a WAN commit (the bandwith is lower for wan) And we
can have a better security, we have no control on what's do on the WAN with
our data and many commit from LAN to WAN are "point of access to private
data" (SSL or not, ssl is not a good security, you can spoof him easily). We
can limit this risk by limitting the data who are sent. 

-Message d'origine-
De : Ryan Schmidt [mailto:subversion-20...@ryandesign.com] 
Envoyé : lundi 2 août 2010 12:04
À : Istace Emmanuel
Cc : 'Tony Sweeney'; users@subversion.apache.org
Objet : Re: SVN "Relay"

On Aug 2, 2010, at 04:54, Istace Emmanuel wrote:

> Ryan Schmidt wrote:
>> Yes, "svnsync" is the software you are looking for. You can read all 
>> about it in the book:
>> 
>> http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.ht
>> ml#svn.reposadmin.maint.replication
>> 
>> You will probably want to sync constantly, not just twice a day.
> 
> Tony Sweeney wrote:
>> Actually, I think he's looking at something more along the lines of 
>> Perforce's P4Proxy server, but for Subversion.
>> 
>> http://www.perforce.com/perforce/doc.current/manuals/p4sag/09_p4p.htm
>> l
>> 
>> Svnsync doesn't help in the event that someone not on his LAN 
>> independently commits to the Subversion server on the WAN, as you'd 
>> need to be able to sync bidirectionally, and I'm not even sure that's
possible.
> 
> To Ryan : Thanks a lot for this, you will make my router happy :D For 
> the replication i will see what's the traffic generated by a constant 
> réplication. This WAN svn service is a temp service, in the next week 
> we will host everything locally and just keep an external svn backup
server.
> 
> To Tony :  It's a possibility but i don't have to do a bidirectionnal
sync.
> But i will have a look at your solution and choose between the two after.

There shouldn't be much difference in bandwidth whether you svnsync twice a
day or immediately after every revision is committed; svnsync replays the
commits in order so they're going to be the same size either way.

No users will be writing to the LAN mirror; all users will be writing to the
master repository on the WAN, even those users who checked out from the LAN
mirror. This is accomplished by setting up a write-through proxy.

Re: SVN "Relay"

2010-08-02 Thread Ryan Schmidt

On Aug 2, 2010, at 04:54, Istace Emmanuel wrote:

> Ryan Schmidt wrote:
>> Yes, "svnsync" is the software you are looking for. You can read all 
>> about it in the book:
>> 
>> http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.ht
>> ml#svn.reposadmin.maint.replication
>> 
>> You will probably want to sync constantly, not just twice a day.
> 
> Tony Sweeney wrote:
>> Actually, I think he's looking at something more along the lines of
>> Perforce's P4Proxy server, but for Subversion. 
>> 
>> http://www.perforce.com/perforce/doc.current/manuals/p4sag/09_p4p.html  
>> 
>> Svnsync doesn't help in the event that someone not on his LAN independently
>> commits to the Subversion server on the WAN, as you'd need to be able to
>> sync bidirectionally, and I'm not even sure that's possible.
> 
> To Ryan : Thanks a lot for this, you will make my router happy :D For the
> replication i will see what's the traffic generated by a constant
> réplication. This WAN svn service is a temp service, in the next week we
> will host everything locally and just keep an external svn backup server. 
> 
> To Tony :  It's a possibility but i don't have to do a bidirectionnal sync.
> But i will have a look at your solution and choose between the two after.

There shouldn't be much difference in bandwidth whether you svnsync twice a day 
or immediately after every revision is committed; svnsync replays the commits 
in order so they're going to be the same size either way.

No users will be writing to the LAN mirror; all users will be writing to the 
master repository on the WAN, even those users who checked out from the LAN 
mirror. This is accomplished by setting up a write-through proxy.

RE: SVN "Relay"

Sorry for the double topic, i'm not a usual user of the mailing list...

-Message d'origine-
De : Istace Emmanuel [mailto:istace.emman...@hotmail.com]
Envoyé : lundi 2 août 2010 11:53
À : 'Tony Sweeney'
Objet : RE: SVN "Relay"

To Ryan : Thanks a lot for this, you will make my router happy :D For the
replication i will see what's the traffic generated by a constant
réplication. This WAN svn service is a temp service, in the next week we
will host everything locally and just keep an external svn backup server. 

To Tony :  It's a possibility but i don't have to do a bidirectionnal sync.
But i will have a look at your solution and choose between the two after.

-Message d'origine-
De : Tony Sweeney [mailto:tswee...@omnifone.com] Envoyé : lundi 2 août 2010
11:49 À : Ryan Schmidt; Istace Emmanuel Cc : users@subversion.apache.org
Objet : RE: SVN "Relay"

Actually, I think he's looking at something more along the lines of
Perforce's P4Proxy server, but for Subversion. 

http://www.perforce.com/perforce/doc.current/manuals/p4sag/09_p4p.html  

Svnsync doesn't help in the event that someone not on his LAN independently
commits to the Subversion server on the WAN, as you'd need to be able to
sync bidirectionally, and I'm not even sure that's possible.

Tony.

> -Original Message-
> From: Ryan Schmidt [mailto:subversion-20...@ryandesign.com]
> Sent: 02 August 2010 10:42
> To: Istace Emmanuel
> Cc: users@subversion.apache.org
> Subject: Re: SVN "Relay"
> 
> 
> On Aug 2, 2010, at 04:38, Istace Emmanuel wrote:
> 
> > Hi, first of all, sorry for my bad english, i'm belgium and
> i'll try to do my best.
> > I have a little problem. I've got a SVN server hosted in
> WAN. In LAN i've got many user who use this service, but, it's really 
> < chatty >. I want to make a SVN Relay into the LAN. This svn server 
> will sync to the WAN server twice per day and my user will connect to 
> the LAN server and not the WAN server. How can i do that ? Is a sync 
> between two svn server possible ?
> 
> Yes, "svnsync" is the software you are looking for. You can read all 
> about it in the book:
> 
> http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.ht
> ml#svn.reposadmin.maint.replication
> 
> You will probably want to sync constantly, not just twice a day.
> 
> 
> 
> __
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit
> http://www.messagelabs.com/email
> __
>

TR: SVN "Relay"

-Message d'origine-
De : Istace Emmanuel [mailto:istace.emman...@hotmail.com] 
Envoyé : lundi 2 août 2010 11:53
À : 'Tony Sweeney'
Objet : RE: SVN "Relay"

To Ryan : Thanks a lot for this, you will make my router happy :D For the
replication i will see what's the traffic generated by a constant
réplication. This WAN svn service is a temp service, in the next week we
will host everything locally and just keep an external svn backup server. 

To Tony :  It's a possibility but i don't have to do a bidirectionnal sync.
But i will have a look at your solution and choose between the two after.

-Message d'origine-
De : Tony Sweeney [mailto:tswee...@omnifone.com] Envoyé : lundi 2 août 2010
11:49 À : Ryan Schmidt; Istace Emmanuel Cc : users@subversion.apache.org
Objet : RE: SVN "Relay"

Actually, I think he's looking at something more along the lines of
Perforce's P4Proxy server, but for Subversion. 

http://www.perforce.com/perforce/doc.current/manuals/p4sag/09_p4p.html  

Svnsync doesn't help in the event that someone not on his LAN independently
commits to the Subversion server on the WAN, as you'd need to be able to
sync bidirectionally, and I'm not even sure that's possible.

Tony.

> -Original Message-
> From: Ryan Schmidt [mailto:subversion-20...@ryandesign.com]
> Sent: 02 August 2010 10:42
> To: Istace Emmanuel
> Cc: users@subversion.apache.org
> Subject: Re: SVN "Relay"
> 
> 
> On Aug 2, 2010, at 04:38, Istace Emmanuel wrote:
> 
> > Hi, first of all, sorry for my bad english, i'm belgium and
> i'll try to do my best.
> > I have a little problem. I've got a SVN server hosted in
> WAN. In LAN i've got many user who use this service, but, it's really 
> < chatty >. I want to make a SVN Relay into the LAN. This svn server 
> will sync to the WAN server twice per day and my user will connect to 
> the LAN server and not the WAN server. How can i do that ? Is a sync 
> between two svn server possible ?
> 
> Yes, "svnsync" is the software you are looking for. You can read all 
> about it in the book:
> 
> http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.ht
> ml#svn.reposadmin.maint.replication
> 
> You will probably want to sync constantly, not just twice a day.
> 
> 
> 
> __
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit
> http://www.messagelabs.com/email
> __
>

RE: SVN "Relay"

2010-08-02 Thread Tony Sweeney

Actually, I think he's looking at something more along the lines of
Perforce's P4Proxy server, but for Subversion. 

http://www.perforce.com/perforce/doc.current/manuals/p4sag/09_p4p.html  

Svnsync doesn't help in the event that someone not on his LAN
independently commits to the Subversion server on the WAN, as you'd need
to be able to sync bidirectionally, and I'm not even sure that's
possible.

Tony.

> -Original Message-
> From: Ryan Schmidt [mailto:subversion-20...@ryandesign.com] 
> Sent: 02 August 2010 10:42
> To: Istace Emmanuel
> Cc: users@subversion.apache.org
> Subject: Re: SVN "Relay"
> 
> 
> On Aug 2, 2010, at 04:38, Istace Emmanuel wrote:
> 
> > Hi, first of all, sorry for my bad english, i'm belgium and 
> i'll try to do my best.
> > I have a little problem. I've got a SVN server hosted in 
> WAN. In LAN i've got many user who use this service, but, 
> it's really < chatty >. I want to make a SVN Relay into the 
> LAN. This svn server will sync to the WAN server twice per 
> day and my user will connect to the LAN server and not the 
> WAN server. How can i do that ? Is a sync between two svn 
> server possible ?
> 
> Yes, "svnsync" is the software you are looking for. You can 
> read all about it in the book:
> 
> http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.ht
> ml#svn.reposadmin.maint.replication
> 
> You will probably want to sync constantly, not just twice a day.
> 
> 
> 
> __
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit 
> http://www.messagelabs.com/email 
> __
>

Re: SVN "Relay"

2010-08-02 Thread Ryan Schmidt


On Aug 2, 2010, at 04:38, Istace Emmanuel wrote:

> Hi, first of all, sorry for my bad english, i’m belgium and i’ll try to do my 
> best.
> I have a little problem. I’ve got a SVN server hosted in WAN. In LAN i’ve got 
> many user who use this service, but, it’s really « chatty ». I want to make a 
> SVN Relay into the LAN. This svn server will sync to the WAN server twice per 
> day and my user will connect to the LAN server and not the WAN server. How 
> can i do that ? Is a sync between two svn server possible ?

Yes, "svnsync" is the software you are looking for. You can read all about it 
in the book:

http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.html#svn.reposadmin.maint.replication

You will probably want to sync constantly, not just twice a day.

SVN "Relay"