SVN and webdav

[Wokash Wolsku]

I am trying to use haproxy to rate control some svn clients which access the 
SVN repro via svn+https.  Some monitoring has thrown up some questions.


  1.When for example doing a checkout or commit of a large number of files 
is this implemented as
 *   1 https request or many
 *   1 web dav action or many?
  2.  from the logs of haproxy (and I am by no means an expert) I see only one 
connect and one https request.

I was hopping to rate limit the clients by IP address and thereafter http 
requests hence to slow down large users so others get a share of the processor. 
 But without the volume of up load being related to http requests I am 
struggling to see how to implement this.

Can anyone offer any advise.

Wocash

Re: SVN and webdav

[Marek Manukjan]

It depends on server and client configuration. See following configuration
attributes

   - SVNAllowBulkUpdates (On, Off, Prefer)
  - Bulk update means that all files will be received in a single
  REPORT request
  - Non-bulk (Skelta) mode, used by default in newer clients, means
  that each file will use its own GET request
  - see
  
https://subversion.apache.org/docs/release-notes/1.8.html#serf-skelta-default
  - see http://svnbook.red-bean.com/en/1.7/svn.ref.mod_dav_svn.conf.html
   - standard Apache configuration attributes KeepAlive, KeepAliveTimeout,
   MaxKeepAliveRequests
  - they control how many HTTP requests can be done in a single
  connection to server
  - https://httpd.apache.org/docs/2.4/mod/core.html#keepalive


On Mon, Oct 5, 2020 at 10:10 AM Wokash Wolsku 
wrote:

> I am trying to use haproxy to rate control some svn clients which access
> the SVN repro via svn+https.  Some monitoring has thrown up some questions.
>
>
>1.   When for example doing a checkout or commit of a large number of
>files is this implemented as
>   1. 1 https request or many
>   2. 1 web dav action or many?
>2. from the logs of haproxy (and I am by no means an expert) I see
>only one connect and one https request.
>
> I was hopping to rate limit the clients by IP address and thereafter http
> requests hence to slow down large users so others get a share of the
> processor.  But without the volume of up load being related to http
> requests I am struggling to see how to implement this.
>
> Can anyone offer any advise.
>
> Wocash
>

Re: SVN and webdav

[Wokash Wolsku]

This is helpful, thank you, however, reading the links especially 
SVNAllowBulkUpdates talks of getting the tree.  What directives control updates 
to the tree.  The situation I face is that some clients have to up load large 
graphics files, and video, these will be one object and presumably report 
request, on first reading that does not appear to be covered by the 
SVNAllowBulkUpdates.

The KeepAlive seem also a corse way to achieve this...

Thanks again.

From: Marek Manukjan 
Sent: 05 October 2020 10:24
To: Wokash Wolsku 
Cc: users@subversion.apache.org 
Subject: Re: SVN and webdav

It depends on server and client configuration. See following configuration 
attributes

  *   SVNAllowBulkUpdates (On, Off, Prefer)
 *   Bulk update means that all files will be received in a single REPORT 
request
 *   Non-bulk (Skelta) mode, used by default in newer clients, means that 
each file will use its own GET request
 *   see 
https://subversion.apache.org/docs/release-notes/1.8.html#serf-skelta-default
 *   see http://svnbook.red-bean.com/en/1.7/svn.ref.mod_dav_svn.conf.html
  *   standard Apache configuration attributes KeepAlive, KeepAliveTimeout, 
MaxKeepAliveRequests
 *   they control how many HTTP requests can be done in a single connection 
to server
 *   https://httpd.apache.org/docs/2.4/mod/core.html#keepalive

On Mon, Oct 5, 2020 at 10:10 AM Wokash Wolsku 
mailto:wokashwol...@outlook.com>> wrote:
I am trying to use haproxy to rate control some svn clients which access the 
SVN repro via svn+https.  Some monitoring has thrown up some questions.


  1.When for example doing a checkout or commit of a large number of files 
is this implemented as
 *   1 https request or many
 *   1 web dav action or many?
  2.  from the logs of haproxy (and I am by no means an expert) I see only one 
connect and one https request.

I was hopping to rate limit the clients by IP address and thereafter http 
requests hence to slow down large users so others get a share of the processor. 
 But without the volume of up load being related to http requests I am 
struggling to see how to implement this.

Can anyone offer any advise.

Wocash

Re: Upgrading visual svn server from 3.7.0 using svn 1.9.7 - how?

[Pavel Lyalyakin]

Hello,

See the download  page which
says that VisualSVN Server 4.2.2
[[[
Includes Apache Subversion 1.10.6.
]]]

If you want to find out the version details of your currently installed
server, use the VisualSVN Server Manager console or check the README.txt
file. See https://stackoverflow.com/a/60887240/761095

Opening cmd.exe and simply running `svn --version` is an incorrect way to
find out the version of Subversion the server is built with. This will only
show the version of the svn.exe client and depending on your %PATH%
variable and current directory can show you the version of some other
Subversion client (e.g., TortoiseSVN). Run the *"%VISUALSVN_SERVER%bin\svn.exe
--version"* command instead - it will show you the version of svn.exe which
comes with VisualSVN Server.

On Fri, Oct 2, 2020 at 8:56 PM Bo Berglund  wrote:

> On Fri, 2 Oct 2020 19:42:03 +0200, Daniel Sahlberg
>  wrote:
>
> >Den fre 2 okt. 2020 kl 18:24 skrev Bo Berglund :
> >
> >> We are using this setup:
> >> - Main server is running on Windows Server 16 Standard using VisualSVN
> >> version 3.7.0, which apparently uses svn 1.9.7
> >>
> >> - The server is using svnsync nightly to synchronize over the Internet
> >> to a mirror SVN server version 1.9.7 running on Ubuntu 18.04 Server on
> >> a different location entirely.
> >> No user operations are allowed on the mirror, it is just a backup.
> >>
> >> My problem is this:
> >> The VisualSVN server is seriously out of date and needs to be
> >> upgraded. In its own management console it suggests upgrading to 4.2.2
> >> but does not say which version of svn will then be running.
> >> In fact it seems like they are intentionally hiding the svn version in
> >> their web pages.. :(
> >>
> >
> >I checked our installation of 4.2.2 and it seems to be running 1.10.6.
> >VisualSVN Server is installing the Subversion command line tools in
> >C:\Program Files\VisualSVN Server\bin so I simply opened cmd.exe and
> >executed svn --version.
> >
> >And I suspect that there might be problems concerning the svnsync
> >> commands if the backup mirror server is not upgraded to the same svn
> >> version, right?
> >>
> >
> >I checked quickly with a brand new Ubuntu 18.4 VM running svn 1.9.7 and
> >svnsync works both if initiated from the Ubuntu box (connecting to
> >VisualSVN Server using https) and if initiated from Windows (using svn+ssh
> >and plink with public keys). Of course, YMMW.
> >
> >
> >> But how do I do that on Ubuntu when I cannot find out which svn
> >> version they use?
> >
> >
> >> Or does it not matter, i.e. can the main and mirror servers be using
> >> different svn versions?
> >>
> >
> >In general use you are free to mix different versions of the server and
> the
> >client so I would assume this also goes for svnsync. And it's not too far
> >between 1.9 and 1.10. Others on the list might be able to give a more
> >detailed answer but why not test it :-)
>
> Thanks! I retrieved the svn version using the same way as you (svn
> --version on command line)...
>
> I will make a test as soon as I have fixed a broken OpenVPN channel to
> the office. It has stopped working even though I can ping the box.
> Unfortunately it sits across the ocean in Texas so it is not so easy.
> And I don't want to risk the upgrade unless I have an extra working
> OpenVPN server on the system.
>
>
> --
> Bo Berglund
> Developer in Sweden
>
>

-- 
With best regards,
Pavel Lyalyakin
VisualSVN Team

Re: Upgrading visual svn server from 3.7.0 using svn 1.9.7 - how?

[Pavel Lyalyakin]

Hello,

On Fri, Oct 2, 2020 at 7:24 PM Bo Berglund  wrote:

> We are using this setup:
> - Main server is running on Windows Server 16 Standard using VisualSVN
> version 3.7.0, which apparently uses svn 1.9.7
>

Usually you just need to download and run the VisualSVN Server installer to
upgrade the server. Please, read the article KB161: Upgrading to VisualSVN
Server 4.2  before
beginning the upgrade. The article includes pre- and post- upgrade
checklists to ensure smooth upgrade.


> - The server is using svnsync nightly to synchronize over the Internet
> to a mirror SVN server version 1.9.7 running on Ubuntu 18.04 Server on
> a different location entirely.
> No user operations are allowed on the mirror, it is just a backup.
>


> My problem is this:
> The VisualSVN server is seriously out of date and needs to be
> upgraded. In its own management console it suggests upgrading to 4.2.2
> but does not say which version of svn will then be running.
> In fact it seems like they are intentionally hiding the svn version in
> their web pages.. :(
>
> And I suspect that there might be problems concerning the svnsync
> commands if the backup mirror server is not upgraded to the same svn
> version, right?
>

>
But how do I do that on Ubuntu when I cannot find out which svn
> version they use?
>
> Or does it not matter, i.e. can the main and mirror servers be using
> different svn versions?
>
> --
> Bo Berglund
> Developer in Sweden
>

--
With best regards,
Pavel Lyalyakin
VisualSVN Team

Re: SVN and webdav

[Marek Manukjan]

I'm not an expert on SVN's HTTP protocol so I don't know if there is
another variant of the Commit command like there are two modes of Update,
but from apache's access log it seems that Commit is done as one initial
POST request, then each file is uploaded as separate PUT request, and the
operation is finalized with MERGE request. Someone else might want to
confirm/disprove this, as it's based purely on observation of the access
log, not in-depth knowledge.

To clear up the difference between using SVNAllowBulkUpdates and KeepAlive,
as they control separate things:

   - SVNAllowBulkUpdates controls how many HTTP *requests will be done in
   total* for Checkout/Update/Switch operation. It doesn't affect Commit,
   as it seems it always uses multiple requests
   - KeepAlive controls how many HTTP *requests will be done per one TCP
   connection* for any operation

>From your initial question I can't quite make out which of the two you want
to actually control (not familiar with haproxy), but both should be
achievable using these configuration attributes.

On Mon, Oct 5, 2020 at 10:49 AM Wokash Wolsku 
wrote:

> This is helpful, thank you, however, reading the links especially
> SVNAllowBulkUpdates talks of getting the tree.  What directives control
> updates to the tree.  The situation I face is that some clients have to up
> load large graphics files, and video, these will be one object and
> presumably report request, on first reading that does not appear to be
> covered by the SVNAllowBulkUpdates.
>
> The KeepAlive seem also a corse way to achieve this...
>
> Thanks again.
> --
> *From:* Marek Manukjan 
> *Sent:* 05 October 2020 10:24
> *To:* Wokash Wolsku 
> *Cc:* users@subversion.apache.org 
> *Subject:* Re: SVN and webdav
>
> It depends on server and client configuration. See following configuration
> attributes
>
>- SVNAllowBulkUpdates (On, Off, Prefer)
>   - Bulk update means that all files will be received in a single
>   REPORT request
>   - Non-bulk (Skelta) mode, used by default in newer clients, means
>   that each file will use its own GET request
>   - see
>   
> https://subversion.apache.org/docs/release-notes/1.8.html#serf-skelta-default
>   - see
>   http://svnbook.red-bean.com/en/1.7/svn.ref.mod_dav_svn.conf.html
>- standard Apache configuration attributes KeepAlive,
>KeepAliveTimeout, MaxKeepAliveRequests
>   - they control how many HTTP requests can be done in a single
>   connection to server
>   - https://httpd.apache.org/docs/2.4/mod/core.html#keepalive
>
>
> On Mon, Oct 5, 2020 at 10:10 AM Wokash Wolsku 
> wrote:
>
> I am trying to use haproxy to rate control some svn clients which access
> the SVN repro via svn+https.  Some monitoring has thrown up some questions.
>
>
>1.   When for example doing a checkout or commit of a large number of
>files is this implemented as
>   1. 1 https request or many
>   2. 1 web dav action or many?
>2. from the logs of haproxy (and I am by no means an expert) I see
>only one connect and one https request.
>
> I was hopping to rate limit the clients by IP address and thereafter http
> requests hence to slow down large users so others get a share of the
> processor.  But without the volume of up load being related to http
> requests I am struggling to see how to implement this.
>
> Can anyone offer any advise.
>
> Wocash
>
>

Re: Upgrading visual svn server from 3.7.0 using svn 1.9.7 - how?

[Bo Berglund]

On Mon, 5 Oct 2020 11:59:53 +0300, Pavel Lyalyakin
 wrote:

>Opening cmd.exe and simply running `svn --version` is an incorrect way to
>find out the version of Subversion the server is built with. This will only
>show the version of the svn.exe client and depending on your %PATH%
>variable and current directory can show you the version of some other
>Subversion client (e.g., TortoiseSVN). Run the *"%VISUALSVN_SERVER%bin\svn.exe
>--version"* command instead - it will show you the version of svn.exe which
>comes with VisualSVN Server.

Here is what I got:

H:\>"%VISUALSVN_SERVER%bin\svn.exe" --version
svn, version 1.9.7 (r1800392)
   compiled Nov 21 2017, 12:52:53 on x86_64-microsoft-windows6.1.7601

and:

H:\>where svn
C:\Program Files\VisualSVN Server\bin\svn.exe

So it is really the server version I have checked.
BTW, no client tools will ever be installed on a Windows Server in our
company, like Tortoise etc.


-- 
Bo Berglund
Developer in Sweden

Re: SVN and webdav

[Daniel Shahaf]

Marek Manukjan wrote on Mon, 05 Oct 2020 10:00 +00:00:
> I'm not an expert on SVN's HTTP protocol so I don't know if there is 
> another variant of the Commit command like there are two modes of 
> Update, but from apache's access log it seems that Commit is done as 
> one initial POST request, then each file is uploaded as separate PUT 
> request, and the operation is finalized with MERGE request. Someone 
> else might want to confirm/disprove this, as it's based purely on 
> observation of the access log, not in-depth knowledge.

This is how commits are performed over http when the client and server
are both sufficiently new.  There _is_ another way to perform commits
over http, used when either the client or server is older.  grep the
source for for "create-txn".

Cheers,

Daniel

> To clear up the difference between using SVNAllowBulkUpdates and 
> KeepAlive, as they control separate things:
>  * SVNAllowBulkUpdates controls how many HTTP *requests will be done in 
> total* for Checkout/Update/Switch operation. It doesn't affect Commit, 
> as it seems it always uses multiple requests
>  * KeepAlive controls how many HTTP *requests will be done per one TCP 
> connection* for any operation
> From your initial question I can't quite make out which of the two you 
> want to actually control (not familiar with haproxy), but both should 
> be achievable using these configuration attributes.
> 
> On Mon, Oct 5, 2020 at 10:49 AM Wokash Wolsku  
> wrote:
> > This is helpful, thank you, however, reading the links especially 
> > SVNAllowBulkUpdates talks of getting the tree.  What directives control 
> > updates to the tree.  The situation I face is that some clients have to up 
> > load large graphics files, and video, these will be one object and 
> > presumably report request, on first reading that does not appear to be 
> > covered by the SVNAllowBulkUpdates.
> > 
> > The KeepAlive seem also a corse way to achieve this...
> > 
> > Thanks again.
> > *From:* Marek Manukjan 
> > *Sent:* 05 October 2020 10:24
> > *To:* Wokash Wolsku 
> > *Cc:* users@subversion.apache.org 
> > *Subject:* Re: SVN and webdav 
> >  
> > It depends on server and client configuration. See following configuration 
> > attributes 
> >  * SVNAllowBulkUpdates (On, Off, Prefer)
> >* Bulk update means that all files will be received in a single REPORT 
> > request
> >* Non-bulk (Skelta) mode, used by default in newer clients, means that 
> > each file will use its own GET request
> >* see 
> > https://subversion.apache.org/docs/release-notes/1.8.html#serf-skelta-default
> >* see http://svnbook.red-bean.com/en/1.7/svn.ref.mod_dav_svn.conf.html
> >  * standard Apache configuration attributes KeepAlive, KeepAliveTimeout, 
> > MaxKeepAliveRequests
> >* they control how many HTTP requests can be done in a single connection 
> > to server
> >* https://httpd.apache.org/docs/2.4/mod/core.html#keepalive
> > 
> > On Mon, Oct 5, 2020 at 10:10 AM Wokash Wolsku  
> > wrote:
> >> I am trying to use haproxy to rate control some svn clients which access 
> >> the SVN repro via svn+https.  Some monitoring has thrown up some questions.
> >> 
> >>  1.   When for example doing a checkout or commit of a large number of 
> >> files is this implemented as 
> >>1. 1 https request or many
> >>2. 1 web dav action or many?
> >>  2. from the logs of haproxy (and I am by no means an expert) I see only 
> >> one connect and one https request.
> >> I was hopping to rate limit the clients by IP address and thereafter http 
> >> requests hence to slow down large users so others get a share of the 
> >> processor.  But without the volume of up load being related to http 
> >> requests I am struggling to see how to implement this.
> >> 
> >> Can anyone offer any advise.
> >> 
> >> Wocash

Re: Upgrading visual svn server from 3.7.0 using svn 1.9.7 - how?

[Pavel Lyalyakin]

Hello Bo,

Yes, your current VisualSVN Server version is linked with Subversion 1.9.7,
and the most recent VisualSVN Server 4.2.2 is linked with Subversion 1.10.6.

Thank you.

On Mon, Oct 5, 2020 at 5:42 PM Bo Berglund  wrote:

> On Mon, 5 Oct 2020 11:59:53 +0300, Pavel Lyalyakin
>  wrote:
>
> >Opening cmd.exe and simply running `svn --version` is an incorrect way to
> >find out the version of Subversion the server is built with. This will
> only
> >show the version of the svn.exe client and depending on your %PATH%
> >variable and current directory can show you the version of some other
> >Subversion client (e.g., TortoiseSVN). Run the
> *"%VISUALSVN_SERVER%bin\svn.exe
> >--version"* command instead - it will show you the version of svn.exe
> which
> >comes with VisualSVN Server.
>
> Here is what I got:
>
> H:\>"%VISUALSVN_SERVER%bin\svn.exe" --version
> svn, version 1.9.7 (r1800392)
>compiled Nov 21 2017, 12:52:53 on x86_64-microsoft-windows6.1.7601
>
> and:
>
> H:\>where svn
> C:\Program Files\VisualSVN Server\bin\svn.exe
>
> So it is really the server version I have checked.
> BTW, no client tools will ever be installed on a Windows Server in our
> company, like Tortoise etc.
>
>
> --
> Bo Berglund
> Developer in Sweden
>
>

-- 
With best regards,
Pavel Lyalyakin
VisualSVN Team