Re: gzip compression (was: Re: Logging Subversion client HTTP requests)

[Manuel Lemos]

Hello,

on 08/01/2011 04:55 AM ehu...@gmail.com said the following:

Please note that many of the responses subversoin sends are already
compressed as binary difference streams against the previous version.
Gzip compression on REPORT responses isn't going to yield an average
factor 5 compression rate.


Last time that I looked, those REPORT responses were XML documents and 
the delta data seemed to be encoded in base64, so that would still leat 
to a lot of compression gains.





Sent from my Nokia phone -Original Message- From: Ryan
Schmidt Sent:  01/08/2011, 09:43 To: Manuel Lemos Cc: Andreas Krey;
Erik Huelsmann; users@subversion.apache.org Subject: Re: gzip
compression (was: Re: Logging Subversion client HTTP requests)



On Aug 1, 2011, at 02:16, Manuel Lemos wrote:


Anyway, another odd thing is that the client always sends a request
header saying it can handle HTTP compression but SubVersion servers
seem to never compress responses.

Given than a typical gzip encoding can compress text data about 5
times, it seemed to me there is a great opportunity to make
SubVersion HTTP server accesses much faster, but that opportunity
is not being addressed because SubVersion HTTP servers do not
compress responses. Again I maybe missing something here.


I believe that's something you can set up in your httpd.conf if
desired. It's a feature of Apache and not specific to (nor
configurable from) Subversion.





--

Regards,
Manuel Lemos

JS Classes - Free ready to use OOP components written in JavaScript
http://www.jsclasses.org/

Re: Logging Subversion client HTTP requests

[Manuel Lemos]

Hello,

on 08/01/2011 05:23 AM Andreas Krey said the following:

On Mon, 01 Aug 2011 04:20:07 +, Manuel Lemos wrote:
...

Been there, tried that, but wireshark does not decrypt SSL traffic
unless you have the server SSL key, which is not the case because the
server is not under my control.


You can man-in-the-middle yourself to see that is going on on the wire.
A bit of openssl/stunnel.


I am not familiar with that technique. Do you recommend any articles to 
learn about it?




Also, the actual traffic within the SSL session should be the same as
for http (modulo things like secure cookies).


Yes, but the SubVersion servers in question do not seem to provide 
non-SSL versions.


--

Regards,
Manuel Lemos

JS Classes - Free ready to use OOP components written in JavaScript
http://www.jsclasses.org/

Re: gzip compression

[Manuel Lemos]

Hello,

on 08/01/2011 04:43 AM Ryan Schmidt said the following:
>> Anyway, another odd thing is that the client always sends a request
>> header saying it can handle HTTP compression but SubVersion servers
>> seem to never compress responses.
>>
>> Given than a typical gzip encoding can compress text data about 5
>> times, it seemed to me there is a great opportunity to make
>> SubVersion HTTP server accesses much faster, but that opportunity
>> is not being addressed because SubVersion HTTP servers do not
>> compress responses. Again I maybe missing something here.
>
> I believe that's something you can set up in your httpd.conf if

Right, but AFAIK I have not seen any SubVersion repository that has 
enabled the gzip compression, not even Google Code, despite these days 
Google is all obcessed with performance.



> desired. It's a feature of Apache and not specific to (nor
> configurable from) Subversion.

Right but this is just an opportunity for SubVersion core developers to 
think about adding gzip/deflate compression built-in mod_dav_svn, 
because if they do not do it, no SubVersion repository is doing it and 
SubVersion servers could be much faster then they are today but they aren't.


I am not sure what is the way to submit improvement suggestions, but if 
any SubVersion core developer is reading this, please consider adding 
gzip/deflate compression built-in mod_dav_svn.



--

Regards,
Manuel Lemos

JS Classes - Free ready to use OOP components written in JavaScript
http://www.jsclasses.org/

Re: Logging Subversion client HTTP requests

[Manuel Lemos]

on 08/01/2011 05:31 AM Andreas Krey said the following:


Anyway, in my implementation I did not do any repeated requests and I
can retrieve the information I want apparently much faster in PHP than
using the svn program. So I wonder if could be missing anything.


It *may* be another opportunity to bash the maturity of libsvn. :-)
I assume the aforementioned protocol change does not just mean eliminating
these redundancies.


I did not intend to bash anybody. It just seemed odd to see those 
apparently needless repeated request, as well never seeing gzip 
compressed responses which could have made SubVersion servers much aste.




Given than a typical gzip encoding can compress text data about 5 times,
it seemed to me there is a great opportunity to make SubVersion HTTP
server accesses much faster, but that opportunity is not being addressed
because SubVersion HTTP servers do not compress responses. Again I maybe
missing something here.


I use ssh port forwarding for svn http:// access anyway, and that can
take care of the compression, even of the headers. Like:

   ssh -L 4080:svnhost:80 wellconnected.host
   svn checkout http://localhost:4080/path/to/repo


I see but that compression only happens inside your ssh tunnel. I am 
sure a direct connection with gzip/deflate compression would be much faster.



--

Regards,
Manuel Lemos

JS Classes - Free ready to use OOP components written in JavaScript
http://www.jsclasses.org/

Re: AW: Logging Subversion client HTTP requests

[Manuel Lemos]

on 08/01/2011 09:55 AM Les Mikesell said the following:

On 8/1/11 2:37 AM, Manuel Lemos wrote:



As I mentioned before, I needed to see traffic to SSL servers, which
you cannot
see with Wireshark unless you have the server private SSL key, which
is not the
case. Thanks for the tip anyway.



If you have read access to a full repository, you should be able to
mirror it to your own instance with svnsync - after which you can set up
a server any way you like.


Right, but I am afraid it would not be the same because I could not 
figure the issues my client has with specific types of SubVersion 
servers but not with svn based servers.



--

Regards,
Manuel Lemos

JS Classes - Free ready to use OOP components written in JavaScript
http://www.jsclasses.org/

Re: Releasing an open source subversion client

[Ryan Schmidt]

On Aug 5, 2011, at 01:47, Rajith Chathunga wrote:

> I'm Rajith Chathunga from university of Moratuwa, Sri Lanka. My team is 
> developing an open source Subversion client as our internship project. It 
> will be a great help that you can provide us some directions to release it. 
> What is the procedure that we should follow to release it?

I'd begin by hosting the source code, issue tracker, mailing lists, etc. on 
public servers. Google Code, github, SourceForge, etc. Or if the University 
already has such infrastructure set up on its own or would like to set such a 
thing up, you can of course host it yourself. You can also announce the URL of 
your project here on the Subversion Users mailing list. This should attract 
attention from interested parties.

More good advice is in the book Producing Open Source Software by Karl Fogel, 
one of the Subversion developers. You can read it for free online, or order a 
print copy:

http://producingoss.com/

Re: 1.7 'svn upgrade' issue on windows vista

[Stephen Meechan]

>> The problem was that the svn upgrade command is case sensitive on the WC 
>> folder name and failed if the
>> name doesn't match exactly. Windows itself is case insensitive and none of 
>> the other svn commands are
>> case sensitive to the WC folder name.
>
>That is correct. The Subversion client, and a Subversion repository, are 
>always case-sensitive,
>regardless of whether your local filesystem is case-sensitive or not. I don't 
>believe this is considered >a bug. So be careful to specify things with the 
>correct case.

2 points:

1. I'm aware the repository itself is case sensitive, but all the other svn 
 commands are case insensitive for the WC name, so "svn upgrade" isn't 
working consistently with them

2. the reported error is misleading and meant spending several hours 
investigating why the working copy couldn't be upgraded. (Which is the reason 
for testing with beta releases)

If it's not a bug, then a an more appropriate error message would be an 
improvement.

Estimation of repository upgrade

[Giulio Troccoli]

I'm working on a plan to upgrade our server from 1.4.6 to 1.6.17.

We have 75 repositories, with an average size of 30MB. They're not big, 
I agree, but I wonder if anyone has any tip on how to estimate how long 
the "svnadmin upgrade" command will take. I mean, will it be a matter or 
minutes or hours?


Also, I'm confused whether I need to run "svnadmin pack" as well, or if 
the upgrade will do it for me. If I need to run, does anyone has an idea 
of how long it could take?


Thanks
Giulio Troccoli

RE: 1.7 'svn upgrade' issue on windows vista

[Bert Huijben]

> -Original Message-
> From: Stephen Meechan [mailto:s...@smeechan.co.uk]
> Sent: vrijdag 5 augustus 2011 1:04
> To: users@subversion.apache.org
> Subject: 1.7 'svn upgrade' issue on windows vista
> 
> While testing out the svn upgrade feature in 1.7 beta 2, on some working
> copies the upgrade failed with an error E155036.
> 
> The problem was that the svn upgrade command is case sensitive on the WC
> folder name and failed if the name doesn't match exactly. Windows itself is
> case insensitive and none of the other svn commands are case sensitive to
> the WC folder name.
> 
> The following test case shows the steps to reproduce the problem using a
> public server:

Thanks for reporting (and the nice reproduction recipe).

I just fixed the issue in r1154165.

(This patch won't make beta 3, but I'll nominate it for backport for 1.7.0)

Bert 

Re: Releasing an open source subversion client

[Nico Kadel-Garcia]

On Fri, Aug 5, 2011 at 6:05 AM, Ryan Schmidt
 wrote:
> On Aug 5, 2011, at 01:47, Rajith Chathunga wrote:
>
>> I'm Rajith Chathunga from university of Moratuwa, Sri Lanka. My team is 
>> developing an open source Subversion client as our internship project. It 
>> will be a great help that you can provide us some directions to release it. 
>> What is the procedure that we should follow to release it?
>
> I'd begin by hosting the source code, issue tracker, mailing lists, etc. on 
> public servers. Google Code, github, SourceForge, etc. Or if the University 
> already has such infrastructure set up on its own or would like to set such a 
> thing up, you can of course host it yourself. You can also announce the URL 
> of your project here on the Subversion Users mailing list. This should 
> attract attention from interested parties.

You might see if you can get it up at Apache, since Subversion itself
is now there. . Sourceforge is less picky and easily supports
Subversion access for users and authors.

But, why, exactly, is your team writing a new client? Would the effort
be better spent on the core Subversion tools themselves, or enhancing
one of the IDE's that support subversion?

Re: Estimation of repository upgrade

[Andy Levy]

On Fri, Aug 5, 2011 at 07:09, Giulio Troccoli
 wrote:
> I'm working on a plan to upgrade our server from 1.4.6 to 1.6.17.
>
> We have 75 repositories, with an average size of 30MB. They're not big, I
> agree, but I wonder if anyone has any tip on how to estimate how long the
> "svnadmin upgrade" command will take. I mean, will it be a matter or minutes
> or hours?
>
> Also, I'm confused whether I need to run "svnadmin pack" as well, or if the
> upgrade will do it for me. If I need to run, does anyone has an idea of how
> long it could take?

I don't believe the upgrade automatically packs the revisions. How
long either of these operations will take depends greatly upon the
repository content and your hardware.

Make a copy of one (or more) of the repositories, and test it for
yourself on another system to get an idea of the time required.

Re: 1.7 'svn upgrade' issue on windows vista

[Stephen Meechan]

thanks Bert, I'll continue with some more testing, with a much faster svn 
update on large working copies this release looks like a very worthwhile 
upgrade.

- Original Message -
From: "Bert Huijben" 
To: "Stephen Meechan" , users@subversion.apache.org
Sent: Friday, 5 August, 2011 12:32:41 GMT +00:00 GMT Britain, Ireland, Portugal
Subject: RE: 1.7 'svn upgrade' issue on windows vista



> -Original Message-
> From: Stephen Meechan [mailto:s...@smeechan.co.uk]
> Sent: vrijdag 5 augustus 2011 1:04
> To: users@subversion.apache.org
> Subject: 1.7 'svn upgrade' issue on windows vista
> 
> While testing out the svn upgrade feature in 1.7 beta 2, on some working
> copies the upgrade failed with an error E155036.
> 
> The problem was that the svn upgrade command is case sensitive on the WC
> folder name and failed if the name doesn't match exactly. Windows itself is
> case insensitive and none of the other svn commands are case sensitive to
> the WC folder name.
> 
> The following test case shows the steps to reproduce the problem using a
> public server:

Thanks for reporting (and the nice reproduction recipe).

I just fixed the issue in r1154165.

(This patch won't make beta 3, but I'll nominate it for backport for 1.7.0)

Bert 

Re: Estimation of repository upgrade

[Mark Phippard]

On Fri, Aug 5, 2011 at 7:09 AM, Giulio Troccoli <
giulio.trocc...@mediatelgroup.co.uk> wrote:

> I'm working on a plan to upgrade our server from 1.4.6 to 1.6.17.
>
> We have 75 repositories, with an average size of 30MB. They're not big, I
> agree, but I wonder if anyone has any tip on how to estimate how long the
> "svnadmin upgrade" command will take. I mean, will it be a matter or minutes
> or hours?
>

svnadmin upgrade runs in literally a few milliseconds.  All it does is
change the format number of the repository.  If you want to fully move to
all of the new repository features in 1.6 you have to do dump/load which
will take longer.


>
> Also, I'm confused whether I need to run "svnadmin pack" as well, or if the
> upgrade will do it for me. If I need to run, does anyone has an idea of how
> long it could take?
>

svnadmin pack is never an automatic feature.  Unless you are really short on
inodes in the volume I would not even recommend running it.

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

RE: Releasing an open source subversion client

[Bob Archer]

> Hi Subversion,
> 
> I'm Rajith Chathunga from university of Moratuwa, Sri Lanka. My team is
> developing an open source Subversion client as our internship project. It will
> be a great help that you can provide us some directions to release it. What is
> the procedure that we should follow to release it?
> 
> Thank you.

If you are going to put a whole team of devs to work on svn why not work with 
the current svn team to identify a feature for the current 1.7 client to 
implement. There is tons of stuff on the 1.7 list that got pushed off to 1.8. 
Or work with the TortoiseSVN team to add a feature to that product if you are 
working in windows.

I just don't see why you want to create YASC.

BOb

Re: Estimation of repository upgrade

[Giulio Troccoli]

On 05/08/11 14:18, Mark Phippard wrote:
On Fri, Aug 5, 2011 at 7:09 AM, Giulio Troccoli 
> wrote:


I'm working on a plan to upgrade our server from 1.4.6 to 1.6.17.

We have 75 repositories, with an average size of 30MB. They're not
big, I agree, but I wonder if anyone has any tip on how to
estimate how long the "svnadmin upgrade" command will take. I
mean, will it be a matter or minutes or hours?


svnadmin upgrade runs in literally a few milliseconds.  All it does is 
change the format number of the repository.  If you want to fully move 
to all of the new repository features in 1.6 you have to do dump/load 
which will take longer.


Wouldn't svnadmin upgrade give me the exact same thing as a dump/load 
cycle? If not, what would be different?




Also, I'm confused whether I need to run "svnadmin pack" as well,
or if the upgrade will do it for me. If I need to run, does anyone
has an idea of how long it could take?


svnadmin pack is never an automatic feature.  Unless you are really 
short on inodes in the volume I would not even recommend running it.




Thanks
Giulio Trocccoli

Re: Estimation of repository upgrade

[Andy Levy]

On Fri, Aug 5, 2011 at 10:11, Giulio Troccoli
 wrote:
>
>
> On 05/08/11 14:18, Mark Phippard wrote:
>>
>> On Fri, Aug 5, 2011 at 7:09 AM, Giulio Troccoli
>> > > wrote:
>>
>>    I'm working on a plan to upgrade our server from 1.4.6 to 1.6.17.
>>
>>    We have 75 repositories, with an average size of 30MB. They're not
>>    big, I agree, but I wonder if anyone has any tip on how to
>>    estimate how long the "svnadmin upgrade" command will take. I
>>    mean, will it be a matter or minutes or hours?
>>
>>
>> svnadmin upgrade runs in literally a few milliseconds.  All it does is
>> change the format number of the repository.  If you want to fully move to
>> all of the new repository features in 1.6 you have to do dump/load which
>> will take longer.
>
> Wouldn't svnadmin upgrade give me the exact same thing as a dump/load cycle?
> If not, what would be different?

svnadmin upgrade upgrades the repository in-place to let you use the
newer features going forward, but they won't be available on older
revisions.

Re: Estimation of repository upgrade

[Mark Phippard]

On Fri, Aug 5, 2011 at 10:11 AM, Giulio Troccoli <
giulio.trocc...@mediatelgroup.co.uk> wrote:

>
>
> On 05/08/11 14:18, Mark Phippard wrote:
>
>  On Fri, Aug 5, 2011 at 7:09 AM, Giulio Troccoli > mediatelgroup.co.uk  > giulio.troccoli@**mediatelgroup.co.uk>>
>> wrote:
>>
>>I'm working on a plan to upgrade our server from 1.4.6 to 1.6.17.
>>
>>We have 75 repositories, with an average size of 30MB. They're not
>>big, I agree, but I wonder if anyone has any tip on how to
>>estimate how long the "svnadmin upgrade" command will take. I
>>mean, will it be a matter or minutes or hours?
>>
>>
>> svnadmin upgrade runs in literally a few milliseconds.  All it does is
>> change the format number of the repository.  If you want to fully move to
>> all of the new repository features in 1.6 you have to do dump/load which
>> will take longer.
>>
>
> Wouldn't svnadmin upgrade give me the exact same thing as a dump/load
> cycle? If not, what would be different?


svnadmin upgrade bumps the repository format number which allows certain new
features to be used.  But there are other things like representation-sharing
that require a full dump/load to get.

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

RE: Estimation of repository upgrade

[Bob Archer]

> On Fri, Aug 5, 2011 at 10:11 AM, Giulio Troccoli
>  wrote:
> 
> 
> On 05/08/11 14:18, Mark Phippard wrote:
> 
> On Fri, Aug 5, 2011 at 7:09 AM, Giulio Troccoli
>  > wrote:
> 
>    I'm working on a plan to upgrade our server from 1.4.6 to 1.6.17.
> 
>    We have 75 repositories, with an average size of 30MB. They're not
>    big, I agree, but I wonder if anyone has any tip on how to
>    estimate how long the "svnadmin upgrade" command will take. I
>    mean, will it be a matter or minutes or hours?
> 
> 
> svnadmin upgrade runs in literally a few milliseconds.  All it does is change 
> the
> format number of the repository.  If you want to fully move to all of the new
> repository features in 1.6 you have to do dump/load which will take longer.
> 
> Wouldn't svnadmin upgrade give me the exact same thing as a dump/load
> cycle? If not, what would be different?
> 
> svnadmin upgrade bumps the repository format number which allows certain
> new features to be used.  But there are other things like representation-
> sharing that require a full dump/load to get.
> 

Has anything been done to improve the speed of a load. Last time I did it, when 
moving from 1.5 to 1.6 it took over 12 hours. And now, two years later the repo 
is probably quite a bit bigger (because we stupidly store binaries in it, I 
might change that when upgrading to 1.7).

BOb

Re: Estimation of repository upgrade

[Nico Kadel-Garcia]

On Fri, Aug 5, 2011 at 10:58 AM, Bob Archer  wrote:

> Has anything been done to improve the speed of a load. Last time I did it, 
> when moving from 1.5 to 1.6 it took over 12 hours. And now, two years later 
> the repo is probably quite a bit bigger (because we stupidly store binaries 
> in it, I might change that when upgrading to 1.7).

You can pre-configure by using svnsync to pre-create a new repository,
and replicate relevant configuration files or scripts manually when
ready to switchover. This is what I've used for migrating between
servers that do not have shared back end storage, especially when
updating operating systems and Subversion releases on the servers.

Re: Estimation of repository upgrade

[Giulio Troccoli]

On 05/08/11 17:11, Nico Kadel-Garcia wrote:

On Fri, Aug 5, 2011 at 10:58 AM, Bob Archer  wrote:


Has anything been done to improve the speed of a load. Last time I did it, when 
moving from 1.5 to 1.6 it took over 12 hours. And now, two years later the repo 
is probably quite a bit bigger (because we stupidly store binaries in it, I 
might change that when upgrading to 1.7).

You can pre-configure by using svnsync to pre-create a new repository,
and replicate relevant configuration files or scripts manually when
ready to switchover. This is what I've used for migrating between
servers that do not have shared back end storage, especially when
updating operating systems and Subversion releases on the servers.


Thanks Nico, I was thinking along those lines. However, the need for an 
upgrade is mostly for bug fixes and the fact that 1.4 is out of support. 
We will not use any (that I know of) of the new feature in 1.6 so I'm 
not even sure now if we need to run an svnadmin upgrade a all.

RE: Estimation of repository upgrade

[Bob Archer]

> On Fri, Aug 5, 2011 at 10:58 AM, Bob Archer  wrote:
> 
> > Has anything been done to improve the speed of a load. Last time I did it,
> when moving from 1.5 to 1.6 it took over 12 hours. And now, two years later
> the repo is probably quite a bit bigger (because we stupidly store binaries in
> it, I might change that when upgrading to 1.7).
> 
> You can pre-configure by using svnsync to pre-create a new repository, and
> replicate relevant configuration files or scripts manually when ready to
> switchover. This is what I've used for migrating between servers that do not
> have shared back end storage, especially when updating operating systems
> and Subversion releases on the servers.

But, doesn't that change the GUID of the repository thus not allowing a 
relocate switch?

BOb

Re: Estimation of repository upgrade

[Nico Kadel-Garcia]

On Fri, Aug 5, 2011 at 12:19 PM, Bob Archer  wrote:
>> On Fri, Aug 5, 2011 at 10:58 AM, Bob Archer  wrote:
>>
>> > Has anything been done to improve the speed of a load. Last time I did it,
>> when moving from 1.5 to 1.6 it took over 12 hours. And now, two years later
>> the repo is probably quite a bit bigger (because we stupidly store binaries 
>> in
>> it, I might change that when upgrading to 1.7).
>>
>> You can pre-configure by using svnsync to pre-create a new repository, and
>> replicate relevant configuration files or scripts manually when ready to
>> switchover. This is what I've used for migrating between servers that do not
>> have shared back end storage, especially when updating operating systems
>> and Subversion releases on the servers.
>
> But, doesn't that change the GUID of the repository thus not allowing a 
> relocate switch?

Until you manually copy over the $repodir/db/uuid file, this is true.
That's one of the "relevant configuraton files" I referred to.

RE: Estimation of repository upgrade

[Bob Archer]

> On Fri, Aug 5, 2011 at 12:19 PM, Bob Archer  wrote:
> >> On Fri, Aug 5, 2011 at 10:58 AM, Bob Archer 
> wrote:
> >>
> >> > Has anything been done to improve the speed of a load. Last time I
> >> > did it,
> >> when moving from 1.5 to 1.6 it took over 12 hours. And now, two years
> >> later the repo is probably quite a bit bigger (because we stupidly
> >> store binaries in it, I might change that when upgrading to 1.7).
> >>
> >> You can pre-configure by using svnsync to pre-create a new
> >> repository, and replicate relevant configuration files or scripts
> >> manually when ready to switchover. This is what I've used for
> >> migrating between servers that do not have shared back end storage,
> >> especially when updating operating systems and Subversion releases on
> the servers.
> >
> > But, doesn't that change the GUID of the repository thus not allowing a
> relocate switch?
> 
> Until you manually copy over the $repodir/db/uuid file, this is true.
> That's one of the "relevant configuraton files" I referred to.

So, are you saying svnsync will be faster than a dump/load?

I didn't know the guid was stored in a file.

BOb

Re: Estimation of repository upgrade

[Mark Phippard]

On Fri, Aug 5, 2011 at 1:19 PM, Bob Archer  wrote:


> > Until you manually copy over the $repodir/db/uuid file, this is true.
> > That's one of the "relevant configuraton files" I referred to.
>
> So, are you saying svnsync will be faster than a dump/load?
>
> I didn't know the guid was stored in a file.
>

svnsync is slower than dump/load.  I think the issue is that you can keep
the old repository online during the process and switch when you are ready.

BTW, why copy a file you are not supposed to when you could just use the
svnadmin setuuid command to make the Repository UUID match between the two
repositories?  No need to involve yourself in the internals of the
repository when there is a public interface to do it for you.

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

RE: Estimation of repository upgrade

[Bob Archer]

> On Fri, Aug 5, 2011 at 1:19 PM, Bob Archer  wrote:
> 
> > Until you manually copy over the $repodir/db/uuid file, this is true.
> > That's one of the "relevant configuraton files" I referred to.
> So, are you saying svnsync will be faster than a dump/load?
> 
> I didn't know the guid was stored in a file.
> 
> svnsync is slower than dump/load.  I think the issue is that you can keep the
> old repository online during the process and switch when you are ready.
> 
> BTW, why copy a file you are not supposed to when you could just use the
> svnadmin setuuid command to make the Repository UUID match between
> the two repositories?  No need to involve yourself in the internals of the
> repository when there is a public interface to do it for you.

Yes, that was my thought too. I will just run the dump/load during the weekend 
again. Also, now that I know you can dump/load in a single step rather than to 
disk then load it shouldn't be so bad.

I do think I will plan to dump filter out all the paths with binaries and move 
them to the file system only or a separate repository that can be purged 
periodically and then use externals to bring them into the WC. I think some 
people here have mentioned a binary library tool that rotates your binaries... 
or I can just add it to my nant scripts like I currently do it with test 
installers.

Thanks,
BOb

Re: Estimation of repository upgrade

[Nico Kadel-Garcia]

On Fri, Aug 5, 2011 at 1:27 PM, Mark Phippard  wrote:
> On Fri, Aug 5, 2011 at 1:19 PM, Bob Archer  wrote:
>
>>
>> > Until you manually copy over the $repodir/db/uuid file, this is true.
>> > That's one of the "relevant configuraton files" I referred to.
>>
>> So, are you saying svnsync will be faster than a dump/load?
>>
>> I didn't know the guid was stored in a file.
>
> svnsync is slower than dump/load.  I think the issue is that you can keep
> the old repository online during the process and switch when you are ready.

And you can start the process a week in advance, run performance tests
on the new server, provide read access to run performance and
regression tests, etc. It's often a much safer way to do a staged
migration.

> BTW, why copy a file you are not supposed to when you could just use the
> svnadmin setuuid command to make the Repository UUID match between the two
> repositories?  No need to involve yourself in the internals of the
> repository when there is a public interface to do it for you.

Good point.

Re: Estimation of repository upgrade

[Erik Huelsmann]

On Fri, Aug 5, 2011 at 7:27 PM, Mark Phippard  wrote:

> On Fri, Aug 5, 2011 at 1:19 PM, Bob Archer  wrote:
>
>
>> > Until you manually copy over the $repodir/db/uuid file, this is true.
>> > That's one of the "relevant configuraton files" I referred to.
>>
>> So, are you saying svnsync will be faster than a dump/load?
>>
>> I didn't know the guid was stored in a file.
>>
>
> svnsync is slower than dump/load.  I think the issue is that you can keep
> the old repository online during the process and switch when you are ready.
>

But there's no difference in running 'checkout' repeatedly, svnsync or
svnadmin dump; all methods can be used concurrently and don't require taking
down the repository. Of course, running during the weekend may help mitigate
the performance effect it may have on users if you start claiming large
amounts of CPU from your server.

Bye,


Erik.

Re: Estimation of repository upgrade

[Les Mikesell]

On 8/5/2011 12:27 PM, Mark Phippard wrote:


 > Until you manually copy over the $repodir/db/uuid file, this is true.
 > That's one of the "relevant configuraton files" I referred to.

So, are you saying svnsync will be faster than a dump/load?

I didn't know the guid was stored in a file.


svnsync is slower than dump/load.  I think the issue is that you can
keep the old repository online during the process and switch when you
are ready.

BTW, why copy a file you are not supposed to when you could just use the
svnadmin setuuid command to make the Repository UUID match between the
two repositories?  No need to involve yourself in the internals of the
repository when there is a public interface to do it for you.


Can you copy from source to dest in one step with the interface (or 
svnsync) like scp or rsync will do for you?


--
  Les Mikesell
lesmikes...@gmail.com

Re: Estimation of repository upgrade

[Erik Huelsmann]

On Fri, Aug 5, 2011 at 7:52 PM, Les Mikesell  wrote:

> On 8/5/2011 12:27 PM, Mark Phippard wrote:
>
>>
>> > Until you manually copy over the $repodir/db/uuid file, this is
>> true.
>> > That's one of the "relevant configuraton files" I referred to.
>>
>>So, are you saying svnsync will be faster than a dump/load?
>>
>>I didn't know the guid was stored in a file.
>>
>>
>> svnsync is slower than dump/load.  I think the issue is that you can
>> keep the old repository online during the process and switch when you
>> are ready.
>>
>> BTW, why copy a file you are not supposed to when you could just use the
>> svnadmin setuuid command to make the Repository UUID match between the
>> two repositories?  No need to involve yourself in the internals of the
>> repository when there is a public interface to do it for you.
>>
>
> Can you copy from source to dest in one step with the interface (or
> svnsync) like scp or rsync will do for you?


You can use svnadmin hotcopy to do that, but it won't have the same effect
as reloading.


Bye,

Erik.

Re: Estimation of repository upgrade

[Mark Phippard]

On Fri, Aug 5, 2011 at 1:52 PM, Les Mikesell  wrote:

> On 8/5/2011 12:27 PM, Mark Phippard wrote:
>
>>
>> > Until you manually copy over the $repodir/db/uuid file, this is
>> true.
>> > That's one of the "relevant configuraton files" I referred to.
>>
>>So, are you saying svnsync will be faster than a dump/load?
>>
>>I didn't know the guid was stored in a file.
>>
>>
>> svnsync is slower than dump/load.  I think the issue is that you can
>> keep the old repository online during the process and switch when you
>> are ready.
>>
>> BTW, why copy a file you are not supposed to when you could just use the
>> svnadmin setuuid command to make the Repository UUID match between the
>> two repositories?  No need to involve yourself in the internals of the
>> repository when there is a public interface to do it for you.
>>
>
> Can you copy from source to dest in one step with the interface (or
> svnsync) like scp or rsync will do for you?
>

Are you talking about the setuuid command?

$ svnadmin help setuuid
setuuid: usage: svnadmin setuuid REPOS_PATH [NEW_UUID]

Reset the repository UUID for the repository located at REPOS_PATH.  If
NEW_UUID is provided, use that as the new repository UUID; otherwise,
generate a brand new UUID for the repository.

Script it however you choose.

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

Re: Estimation of repository upgrade

[Les Mikesell]

On 8/5/2011 12:33 PM, Bob Archer wrote:



Until you manually copy over the $repodir/db/uuid file, this is true.
That's one of the "relevant configuraton files" I referred to.

So, are you saying svnsync will be faster than a dump/load?

I didn't know the guid was stored in a file.

svnsync is slower than dump/load.  I think the issue is that you can keep the
old repository online during the process and switch when you are ready.

BTW, why copy a file you are not supposed to when you could just use the
svnadmin setuuid command to make the Repository UUID match between
the two repositories?  No need to involve yourself in the internals of the
repository when there is a public interface to do it for you.


Yes, that was my thought too. I will just run the dump/load during the weekend 
again. Also, now that I know you can dump/load in a single step rather than to 
disk then load it shouldn't be so bad.


Svnsync can be run anytime without bothering the working repo very much 
so you can have most of the contents in the new one well ahead of time 
and repeated runs pick up where it left off.  You just need to make sure 
that no commits are done while you make the final run before the cutover 
- which can be very quick.



I do think I will plan to dump filter out all the paths with binaries and move 
them to the file system only or a separate repository that can be purged 
periodically and then use externals to bring them into the WC. I think some 
people here have mentioned a binary library tool that rotates your binaries... 
or I can just add it to my nant scripts like I currently do it with test 
installers.


If you come up with a good scheme for this that doesn't interfere too 
much with using externals to pull in component libraries without 
recompiling, please post it.


--
  Les Mikesell
   lesmikes...@gmail.com

Re: Estimation of repository upgrade

[Mark Phippard]

On Fri, Aug 5, 2011 at 1:51 PM, Erik Huelsmann  wrote:

>
> On Fri, Aug 5, 2011 at 7:27 PM, Mark Phippard  wrote:
>
>> On Fri, Aug 5, 2011 at 1:19 PM, Bob Archer  wrote:
>>
>>
>>> > Until you manually copy over the $repodir/db/uuid file, this is true.
>>> > That's one of the "relevant configuraton files" I referred to.
>>>
>>> So, are you saying svnsync will be faster than a dump/load?
>>>
>>> I didn't know the guid was stored in a file.
>>>
>>
>> svnsync is slower than dump/load.  I think the issue is that you can keep
>> the old repository online during the process and switch when you are ready.
>>
>
> But there's no difference in running 'checkout' repeatedly, svnsync or
> svnadmin dump; all methods can be used concurrently and don't require taking
> down the repository. Of course, running during the weekend may help mitigate
> the performance effect it may have on users if you start claiming large
> amounts of CPU from your server.
>

Anytime the entire time it takes to dump/load a repository exceeds the
amount of time you can reasonably block writes to the old repository, it is
beneficial to be able to use svnsync.  When using svnsync, it can take as
long as it needs to because you have total control over the switchover and
can do it with minimal downtime.  But the actual time to do svnsync is
generally longer than dump/load.  My point was only that you do not use
svnsync because it is faster, you do it because you can better control and
minimize the downtime.

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

RE: Estimation of repository upgrade

[Bob Archer]

> On Fri, Aug 5, 2011 at 1:51 PM, Erik Huelsmann  wrote:
> 
> On Fri, Aug 5, 2011 at 7:27 PM, Mark Phippard 
> wrote:
> On Fri, Aug 5, 2011 at 1:19 PM, Bob Archer  wrote:
> 
> > Until you manually copy over the $repodir/db/uuid file, this is true.
> > That's one of the "relevant configuraton files" I referred to.
> So, are you saying svnsync will be faster than a dump/load?
> 
> I didn't know the guid was stored in a file.
> 
> svnsync is slower than dump/load.  I think the issue is that you can keep the
> old repository online during the process and switch when you are ready.
> 
> But there's no difference in running 'checkout' repeatedly, svnsync or
> svnadmin dump; all methods can be used concurrently and don't require
> taking down the repository. Of course, running during the weekend may
> help mitigate the performance effect it may have on users if you start
> claiming large amounts of CPU from your server.
> 
> Anytime the entire time it takes to dump/load a repository exceeds the
> amount of time you can reasonably block writes to the old repository, it is
> beneficial to be able to use svnsync.  When using svnsync, it can take as long
> as it needs to because you have total control over the switchover and can do
> it with minimal downtime.  But the actual time to do svnsync is generally
> longer than dump/load.  My point was only that you do not use svnsync
> because it is faster, you do it because you can better control and minimize
> the downtime.

I'm never sure... should I dump with 1.6 binaries and then load with 1.7 
binaries. Or, upgrade the binaries and then dump and load with the most recent 
binaries? Or does it not matter since the dump file format is the same for all 
1.x versions?

BOb

Re: Estimation of repository upgrade

[Mark Phippard]

On Fri, Aug 5, 2011 at 2:12 PM, Bob Archer  wrote:

> > On Fri, Aug 5, 2011 at 1:51 PM, Erik Huelsmann  wrote:
> >
> > On Fri, Aug 5, 2011 at 7:27 PM, Mark Phippard 
> > wrote:
> > On Fri, Aug 5, 2011 at 1:19 PM, Bob Archer  wrote:
> >
> > > Until you manually copy over the $repodir/db/uuid file, this is true.
> > > That's one of the "relevant configuraton files" I referred to.
> > So, are you saying svnsync will be faster than a dump/load?
> >
> > I didn't know the guid was stored in a file.
> >
> > svnsync is slower than dump/load.  I think the issue is that you can keep
> the
> > old repository online during the process and switch when you are ready.
> >
> > But there's no difference in running 'checkout' repeatedly, svnsync or
> > svnadmin dump; all methods can be used concurrently and don't require
> > taking down the repository. Of course, running during the weekend may
> > help mitigate the performance effect it may have on users if you start
> > claiming large amounts of CPU from your server.
> >
> > Anytime the entire time it takes to dump/load a repository exceeds the
> > amount of time you can reasonably block writes to the old repository, it
> is
> > beneficial to be able to use svnsync.  When using svnsync, it can take as
> long
> > as it needs to because you have total control over the switchover and can
> do
> > it with minimal downtime.  But the actual time to do svnsync is generally
> > longer than dump/load.  My point was only that you do not use svnsync
> > because it is faster, you do it because you can better control and
> minimize
> > the downtime.
>
> I'm never sure... should I dump with 1.6 binaries and then load with 1.7
> binaries. Or, upgrade the binaries and then dump and load with the most
> recent binaries? Or does it not matter since the dump file format is the
> same for all 1.x versions?
>

For the most part it does not matter for the reason you state.  An exception
is for people using BDB.  They should always dump using their current
binaries.  If the new binaries do not have BDB support or include a newer
version of BDB they will not be able to dump the old repository.  For FSFS
it does not matter since all new versions can read all old versions fine.

I think dump of fsfs might be a little faster in 1.7 due to some of the
caching changes.  So it might make sense to try it with new binaries.

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

Re: RE: Releasing an open source subversion client

[Stephen Connolly]

perhaps they are writing a new java client under a more friendly license
than svnkit... otherwise yeah i agree that it smells like a complete waste
of time and effort.

- Stephen

---
Sent from my Android phone, so random spelling mistakes, random nonsense
words and other nonsense are a direct result of using swype to type on the
screen
On 5 Aug 2011 15:05, "Bob Archer"  wrote:
>> Hi Subversion,
>>
>> I'm Rajith Chathunga from university of Moratuwa, Sri Lanka. My team is
>> developing an open source Subversion client as our internship project. It
will
>> be a great help that you can provide us some directions to release it.
What is
>> the procedure that we should follow to release it?
>>
>> Thank you.
>
> If you are going to put a whole team of devs to work on svn why not work
with the current svn team to identify a feature for the current 1.7 client
to implement. There is tons of stuff on the 1.7 list that got pushed off to
1.8. Or work with the TortoiseSVN team to add a feature to that product if
you are working in windows.
>
> I just don't see why you want to create YASC.
>
> BOb
>

Re: UNS: Re: Logging Subversion client HTTP requests

[Andreas Krey]

On Fri, 05 Aug 2011 05:15:24 +, Manuel Lemos wrote:
...
> >   ssh -L 4080:svnhost:80 wellconnected.host
> >   svn checkout http://localhost:4080/path/to/repo
> 
> I see but that compression only happens inside your ssh tunnel. I am 
> sure a direct connection with gzip/deflate compression would be much faster.

I'm not sure. The svn tunnel wil also compress the headers, and use
previous 'coding adaptions' for subsequent requests, while the gzip
reply compression starts anew for each reply?

Andreas

-- 
"Totally trivial. Famous last words."
From: Linus Torvalds 
Date: Fri, 22 Jan 2010 07:29:21 -0800