Re: scp/sftp performance myths
hmm, on Sun, Oct 30, 2005 at 04:04:52PM -0500, Nick Holland said that
> I find that interesting, too.
> I was just explaining to my GF's six-year-old niece yesterday that you
> shouldn't believe everything someone says.
hello Nick, thanks for the tip :)
you surely realize you are 'someone' as well? ;-)))
(just to poke you a bit, i certainly don't believe everything anyone says)
> Been doing some interesting tests recently...
> scp'ing large (100M+ files) from a Celron 566 to a PIII-750 went at
> about 4MB/s, using fxp cards on both sides. Somewhat less than half
> wire speed. Room for improvement, certainly, but not three times. And
> that's on two-generation old hardware! (and several switches, a router,
> and a firewall between them)
yes, i was expecting someone coming up with the hardware side,
esp. the NIC's. i do not doubt these facts, crappy NIC's have
crappy throughputs.
but.
i normally get 2.5-3MB/s on my lan scp-ing, where the router is a
cpu0: Intel Celeron ("GenuineIntel" 686-class, 128KB L2 cache) 375 MHz
but on the same box 6-7MB/s copying thru samba. ftp the same.
yuck. i enabled it just to make some tests. (i dont even dare
to use the word benchmark: it is not, just some silly end-user tests).
so i don't believe all is hardware, certainly the cards can do
better.
here is the thing: as i said before, noone excepts the same of
scp as of samba or ftp. i write these mails simply because
i don't know what to expect. what could be a normal transfer
rate for my machine? is it my processor that can't do more?
if yes, wouldn't it make sense after all making a cipher=none
option for the data part? ssh is _the_ standard, and it openly
aims to replace ftp. will we have to turn on ftp just to
transfer (3x as fast) some big files which are not confident
and turn it off again after?
also, what do other people say, could someone test with high
end machines where CPU is not an issue? where does ssh top out?
and if it was only a CPU issue, the HPN-SSH couldn't make the
claims they make. even if they are not true in all cases ;)
what about tunnels? maybe around rsync? what's the transfer
rate there? is it the same as scp? or some other tunneling.
could anybody comment on this?
> > i think it would be very nice to have a performance page on the openssh
> > site describing what should be expected, what is "normal" and the
> > intended performance of ssh to clear up possible misunderstandings.
> > (like mine here)
>
> too many variables.
true. i did not think this over.
but the numbers don't have to be absolute, more like informative...
> Oh, and OpenSSH is very multi-platform...again, more variables.
well, isn't ftp? i know the RNG of a particular system plays
a big role, but this is something which could be tested by
cipher=none ;-)
-f
--
name a psychological rock group? pink freud!
Re: scp/sftp performance myths
On Sun, Oct 30, 2005 at 02:41:25PM +0100, frantisek holop wrote: > hi there, > > poking around in the HP ssh docs, one can see the following in the FAQ: > > Q: How is the performance of HP-UX Secure Shell? > A: Compared with conventional file transfer methods, the scp command >is 2 - 3 times slower than rcp, and sftp is 2 to 3 times slower than >ftp. This is because HP-UX Secure Shell authenticates both the server >and users, and encrypts both the data and the password. In addition, >HP recommends you use the /dev/random device on your system to >significantly speed up program initialization. > > i find it interesting that most of the user community perceives > scp/sftp multiple times slower then their not encrypted counterparts. > > now, not taking into consideration the HP-UX itself is a bottleneck > on its own (not mentioning their RNG interface) i think some of us prngd also helps if you can't get a real /dev/random for your system, or can't afford downtime to install the device driver. > agrees that scp/sftp is "kind of slower" when it comes to bulk data > transfer. nobody expects scp to be as fast as samba or ftp of course, > the encryption has a great overhead, especially for older machines > (which my local network router is) Selecting a fast cipher/mac pair can make a significant difference. Of the standard ones, arcfour and hmac-md5-96 are usually fastest. > but a couple of months ago a link appeared here describing a HPN > (as in hihg performance enabled) ssh patch. i kept that mail for > a very long time because i was very much interested in the answers > of the ssh developers, but there was none. and so i assumed it must > be rubish or something. It's not rubbish, but it only helps in certain situations (ie when you have a "long, fat pipe"). The breakpoint is when the amount of data in the pipe gets to about 64kb, which is a RTT of about 5ms on a 100Mb/s network or 50ms on a 10Mb/s one. > so before anyone tags this mail as a trolling flamebait > (which it is not), i just would like to ask > -have others tried HPN-SSH? > -have ssh developers tried it? I have. The current one has a couple of problems: the main one being that it is slower in situations where you have more bandwidth than CPU (10% to 25% in my experiments) and uses more CPU and memory under most conditions. There's no fundamental reason why this should be so since you're not doing any more work, just making better use of the available bandwidth. It also has a few stylistic issues (duplicated code inline rather than in its own function, some slightly convoluted logic, unnecessary changes in scp and use of sizeof(somebuffer) for read limits where the somebuffer is unrelated but just happens to be the right size). I've proposed some changes to address these, but since I don't have a suitable link to test it on (and attempts to simulate one have given inconsistent results) I can't tell if my changes still work as expected. The PSC folks were going to test them but that was a few weeks ago and I've not heard from them since. Most of the discusstion was over on the openssh list. Code review with some proposed changes: http://marc.theaimsgroup.com/?l=openssh-unix-dev&m=112316226728255 Thread on performance issues: http://marc.theaimsgroup.com/?l=openssh-unix-dev&m=112884429502318 The patch attempting to address this is not in the thread: http://bugzilla.mindrot.org/attachment.cgi?id=1013 > i think it would be very nice to have a performance page on the openssh > site describing what should be expected, what is "normal" and the > intended performance of ssh to clear up possible misunderstandings. > (like mine here) http://www.openssh.com/faq.html#3.3 covers authentication times but not throughput. -- Darren Tucker (dtucker at zip.com.au) GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4 37C9 C982 80C7 8FF4 FA69 Good judgement comes with experience. Unfortunately, the experience usually comes from bad judgement.
Re: scp/sftp performance myths
frantisek holop wrote: > hi there, > > poking around in the HP ssh docs, one can see the following in the FAQ: > > Q: How is the performance of HP-UX Secure Shell? > A: Compared with conventional file transfer methods, the scp command >is 2 - 3 times slower than rcp, and sftp is 2 to 3 times slower than >ftp. This is because HP-UX Secure Shell authenticates both the server >and users, and encrypts both the data and the password. In addition, >HP recommends you use the /dev/random device on your system to >significantly speed up program initialization. > > i find it interesting that most of the user community perceives > scp/sftp multiple times slower then their not encrypted counterparts. I find that interesting, too. I was just explaining to my GF's six-year-old niece yesterday that you shouldn't believe everything someone says. Been doing some interesting tests recently... scp'ing large (100M+ files) from a Celron 566 to a PIII-750 went at about 4MB/s, using fxp cards on both sides. Somewhat less than half wire speed. Room for improvement, certainly, but not three times. And that's on two-generation old hardware! (and several switches, a router, and a firewall between them) scp'ing the same large files from the same PIII-750 to an AMD64 3000 processor on the same subnet (though with a LOT of switches between them) managed over 8MBs (sk(4) chip on the amd64, 100Mbps network). Not going to get much better than that. (Well..actually, I *did* get impatient, as there was several hundred gig to transfer, so I pulled the disk out of the PIII and put it in the amd64 and did a disk-to-disk copy). > i think it would be very nice to have a performance page on the openssh > site describing what should be expected, what is "normal" and the > intended performance of ssh to clear up possible misunderstandings. > (like mine here) too many variables. I'd like to grab another amd64 system and a gigabit switch and try my test again, but on modern HW, you should be moving a fair chunk of data. There are some things you should just test yourself, and find your own bottlenecks. BTW: that PIII-750 had a very slow disk system for its age -- UDMA2. The cables were way too long to run at a more respectable rate. Note the difference in the network. And so on and so on. Oh, and OpenSSH is very multi-platform...again, more variables. The people complaining that they didn't get the "expected" performance they saw on such a page would be a never ending nightmare. For example, when I first started writing this, I forgot that the my two test cases involved one common machine, but two very different network paths... Nick.
Re: scp/sftp performance myths
frantisek holop wrote: so before anyone tags this mail as a trolling flamebait (which it is not), i just would like to ask -have others tried HPN-SSH? -have ssh developers tried it? -or simply, has ssh hit its performance limit and can't get any better? the "HPN" patch greatly improves throughput for long and fat (high bandwidth, high latency) connections, but slows things down significantly for low latency connections. One of the OpenSSH developers saw around a 15% slowdown on LAN to LAN copies. It needs work. -d
Re: scp/sftp performance myths
On Sun, 30 Oct 2005, Matthew Weigel wrote: SNIP > HPN-SSH improves OpenSSH performance in situations that you and I don't > deal with. Maybe I'm mistaken... do you have an OC-3 connection you're > trying to scp files across? If you are dealing with T-1s and 100Mbps nope, but will OC-48 or 10G suffice? ;-) I went to his presentation at SC last year in Pittsburgh, not sure if the principle investigator is going to be in Seattle for this years SC. For what it's worth, anyone else going to be at SC this year? I assist in setup and teardown of the ASC booth every year, well at least for the last 5 years. g.day diana
Re: scp/sftp performance myths
frantisek holop wrote: but a couple of months ago a link appeared here describing a HPN (as in hihg performance enabled) ssh patch. i kept that mail for a very long time because i was very much interested in the answers of the ssh developers, but there was none. and so i assumed it must be rubish or something. It's not, but there's not much we can tell you that isn't on the PSC site. In fact, most of my reply is going to be telling you things from their site. The shortest way to say it is: you're barking up the wrong tree. HPN-SSH improves OpenSSH performance in situations that you and I don't deal with. Maybe I'm mistaken... do you have an OC-3 connection you're trying to scp files across? If you are dealing with T-1s and 100Mbps LANs (and maybe Gigabit LANs, I'm not sure), then HPN-SSH doesn't matter to you. It's unfortunate that the patch has not yet been incorporated, but if *you* want faster performance for *your* servers, use faster processors and crypto accelerators, because that's what makes SSH slow for *you*. -- Matthew Weigel
Re: scp/sftp performance myths
On 10/30/05, frantisek holop <[EMAIL PROTECTED]> wrote: > but a couple of months ago a link appeared here describing a HPN > (as in hihg performance enabled) ssh patch. i kept that mail for > a very long time because i was very much interested in the answers > of the ssh developers, but there was none. and so i assumed it must > be rubish or something. > > > so before anyone tags this mail as a trolling flamebait > (which it is not), i just would like to ask > -have others tried HPN-SSH? > -have ssh developers tried it? > -or simply, has ssh hit its performance limit and can't get any better? my understanding is that the patch works, but it's not "right".
scp/sftp performance myths
hi there, poking around in the HP ssh docs, one can see the following in the FAQ: Q: How is the performance of HP-UX Secure Shell? A: Compared with conventional file transfer methods, the scp command is 2 - 3 times slower than rcp, and sftp is 2 to 3 times slower than ftp. This is because HP-UX Secure Shell authenticates both the server and users, and encrypts both the data and the password. In addition, HP recommends you use the /dev/random device on your system to significantly speed up program initialization. i find it interesting that most of the user community perceives scp/sftp multiple times slower then their not encrypted counterparts. now, not taking into consideration the HP-UX itself is a bottleneck on its own (not mentioning their RNG interface) i think some of us agrees that scp/sftp is "kind of slower" when it comes to bulk data transfer. nobody expects scp to be as fast as samba or ftp of course, the encryption has a great overhead, especially for older machines (which my local network router is) but a couple of months ago a link appeared here describing a HPN (as in hihg performance enabled) ssh patch. i kept that mail for a very long time because i was very much interested in the answers of the ssh developers, but there was none. and so i assumed it must be rubish or something. so before anyone tags this mail as a trolling flamebait (which it is not), i just would like to ask -have others tried HPN-SSH? -have ssh developers tried it? -or simply, has ssh hit its performance limit and can't get any better? i think it would be very nice to have a performance page on the openssh site describing what should be expected, what is "normal" and the intended performance of ssh to clear up possible misunderstandings. (like mine here) -f -- i'm so broke i can't even pay attention.
