Re: Expected speed of commit over HTTP?

2017-07-07 Thread Paul Hammant
Great insights, Stefan. So 'cp' with that sync, timed as you outline is averaging 3x slower than the cp as I had it before. Thus, that same cp with sync tacked on, is 3x faster that the curl PUT over HTTP. That's within the place where any speedup spent on the handoff of between apache modules w

Re: Expected speed of commit over HTTP?

2017-07-07 Thread Stefan Fuhrmann
On 07.07.2017 01:10, Paul Hammant wrote: With autorevision set to 'on' and curl: Reverence speed for boot drive to USB3 spinning platter 4TB thing: paul@paul-HiBox:~$ time cp /home/paul/clientDir/seven /media/paul/sg4t/sevenb real0m1.539s That isn't exactly accurate - you write t

Re: Expected speed of commit over HTTP?

2017-07-07 Thread Branko Čibej
On 07.07.2017 12:10, Paul Hammant wrote: > ~/.subversion/servers contains a line that's tunable, right: > > # http-compression = no > > And per comment, tack on: > > --config-option=servers:global:http-compression=no . > > I'll report back the results. > > > How is Apache and its modules or

Re: Expected speed of commit over HTTP?

2017-07-07 Thread Paul Hammant
~/.subversion/servers contains a line that's tunable, right: # http-compression = no And per comment, tack on: --config-option=servers:global:http-compression=no . I'll report back the results. How is Apache and its modules organized? As separate processes - with TCP/IP between each?

Re: Expected speed of commit over HTTP?

2017-07-07 Thread Johan Corveleyn
On Fri, Jul 7, 2017 at 8:54 AM, Jacek Materna wrote: > Paul, > > Got back from Ops. We don't have anything "special" setup other than what > already been mentioned. Overall the defining metrics for commit performance > are: ssh vs https, rtt on the network path and "randomness" of the blobs > goin

Re: Expected speed of commit over HTTP?

2017-07-06 Thread Jacek Materna
Paul, Got back from Ops. We don't have anything "special" setup other than what already been mentioned. Overall the defining metrics for commit performance are: ssh vs https, rtt on the network path and "randomness" of the blobs going up. No magic params on our end. We have it tuned the max v

Re: Expected speed of commit over HTTP?

2017-07-06 Thread Paul Hammant
With autorevision set to 'on' and curl: Reverence speed for boot drive to USB3 spinning platter 4TB thing: paul@paul-HiBox:~$ time cp /home/paul/clientDir/seven /media/paul/sg4t/sevenb real 0m1.539s Create a new 501MB file on Svn Server: paul@paul-HiBox:~$ time curl -u paul:myPassword http://

Re: Expected speed of commit over HTTP?

2017-07-06 Thread Philip Martin
Johan Corveleyn writes: >> On my machine: >> >> dd: 3.3 sec >> svn commit: 37 sec >> svnmucc commit: 26 sec >> curl commit: 9.5 sec > [ cc += Stefan Fuhrman, who might have some more ideas about this ] > > If your SVN server is version 1.9 with FSFS backend, your

Re: Expected speed of commit over HTTP?

2017-07-06 Thread Johan Corveleyn
On Thu, Jul 6, 2017 at 4:10 PM, Philip Martin wrote: > Paul Hammant writes: > >> I'm making each revision with.. >> >> dd if=/dev/zero bs=1M count=500 2>/dev/null > >> path/to/file/under/versionControl.dat >> >> .. which is random enough to completely thwart delta analysis of the file. >> T

Re: Expected speed of commit over HTTP?

2017-07-06 Thread Philip Martin
Philip Martin writes: > Paul Hammant writes: > >> I'm making each revision with.. >> >> dd if=/dev/zero bs=1M count=500 2>/dev/null > >> path/to/file/under/versionControl.dat >> >> .. which is random enough to completely thwart delta analysis of the file. >> That's slow enough in itself, b

Re: Expected speed of commit over HTTP?

2017-07-06 Thread Philip Martin
Paul Hammant writes: > I'm making each revision with.. > > dd if=/dev/zero bs=1M count=500 2>/dev/null > > path/to/file/under/versionControl.dat > > .. which is random enough to completely thwart delta analysis of the file. > That's slow enough in itself, but I'm only checking the time of t

Re: Expected speed of commit over HTTP?

2017-07-06 Thread Jacek Materna
Let me go do some digging with the Ops team- On Thu, Jul 6, 2017 at 2:31 PM, Paul Hammant wrote: > Jacek, Can you share any config settings for mod_dav that might be areas > to experiment with ? > > And, yes, just because the file commit has no discernible deltas at all, > doesn't mean that Svn

Re: Expected speed of commit over HTTP?

2017-07-06 Thread Paul Hammant
Jacek, Can you share any config settings for mod_dav that might be areas to experiment with ? And, yes, just because the file commit has no discernible deltas at all, doesn't mean that Svn doesn't attempt to make its own determination. I wasn't using .bin suffixes as it happens but I'll bet there

Re: Expected speed of commit over HTTP?

2017-07-06 Thread Paul Hammant
I'm making each revision with.. dd if=/dev/zero bs=1M count=500 2>/dev/null > path/to/file/under/versionControl.dat .. which is random enough to completely thwart delta analysis of the file. That's slow enough in itself, but I'm only checking the time of the commit. - Paul On Thu, Jul 6,

Re: Expected speed of commit over HTTP?

2017-07-06 Thread Jacek Materna
We've say 2-3x speed decrease using HTTP vs SSH in our cloud. With SSH and some tuning we are getting close to the IO rate on the end server of any size. HTTP/mod_dav and RTT is usually the culprit in our cases, on a LAN that issues collapses. Likely server side processing of the delta's? -jacek

Re: Expected speed of commit over HTTP?

2017-07-06 Thread Branko Čibej
On 06.07.2017 13:54, Paul Hammant wrote: > For something that's 500MB in size (random binary data) I'm > experiencing commits taking > 10x longer than a straight copy to the drive the Svn repo is on. > > Both timings are on the same Ubuntu 17.04 machine, with the boot drive > being the starting