Re: [opensuse] [SLE] Slow transfers from Linux Server
John Andersen wrote: My point was, that without testing a samba or nfs transfer you have no way of judging the load imposed by scp. I kind of wish there was a flag to tell scp to negotiate the password in a secure way, but *not* to encrypt the transfer. Often, when I'm copying files over a local network, I don't want or need the encryption overhead. But scp is so convenient for doing copies compared to the trouble of setting up an NFS mount (and then dealing with processes hanging in the D state every time the server is down.) -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [opensuse] [SLE] Slow transfers from Linux Server
But scp is so convenient for doing copies compared to the trouble of setting up an NFS mount (and then dealing with processes hanging in the D state every time the server is down.) -- I use rsync and ssh to backup our home directories and the initial transfer of my folder was about 2.0+ GB and (I am not completely sure of the time) less than 5 minutes to transfer. I will time the next complete transfer. note: That transfer was from two different laptops to the lan by way of wireless connection. -- John Registered Linux User 263680, get counted at http://counter.li.org -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [opensuse] [SLE] Slow transfers from Linux Server
On Friday 16 March 2007, David Brodbeck wrote: John Andersen wrote: My point was, that without testing a samba or nfs transfer you have no way of judging the load imposed by scp. I kind of wish there was a flag to tell scp to negotiate the password in a secure way, but *not* to encrypt the transfer. Often, when I'm copying files over a local network, I don't want or need the encryption overhead. But scp is so convenient for doing copies compared to the trouble of setting up an NFS mount (and then dealing with processes hanging in the D state every time the server is down.) Yup, that and the permissions and uid problems are a headache. I end up using sftp/scp for a lot of stuff, but once I put samba on a machine its easier to use smb.cifs, and its plenty fast enough over a local net. -- _ John Andersen pgpet0RjIPj29.pgp Description: PGP signature
Re: [opensuse] [SLE] Slow transfers from Linux Server
On Thursday 15 March 2007 06:48, John Andersen wrote: On Wednesday 14 March 2007, Anders Johansson wrote: Just in case you're still interested, I'm doing this now, and I'm getting a constant data rate of over 10MB/s (in real data, not bits over the wire). This means a 100MB file transfers in 9 seconds, or 1024MB in 1 minute 34 seconds. All using scp I don't doubt that Anders, that performance is quite acceptable. My point was, that without testing a samba or nfs transfer you have no way of judging the load imposed by scp. There is a substantial CPU load, to be sure. My 2GHz Celeron was at ~75% throughout the transfer. A slower CPU would have spiked, causing a slowdown in the transfer The OP did those tests, and his 35minute transfer with ssh dropped to 5 minutes. He suspects a faulty ssh client, as do I, because that much difference it way out of line with what I would expect. Could be. But I'd be interested in knowing what hardware is involved, and what the system load looked like during the transfer -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [opensuse] [SLE] Slow transfers from Linux Server
On Tuesday 13 March 2007 21:14, John Andersen wrote: Have you tried to move a 650meg iso across nfs, and then do the same move across ssh from and to the same source/destination? I think you will find that on local networks where nothing is less than 100meg that ssh is quite a bit slower than a well tuned nfs. Just in case you're still interested, I'm doing this now, and I'm getting a constant data rate of over 10MB/s (in real data, not bits over the wire). This means a 100MB file transfers in 9 seconds, or 1024MB in 1 minute 34 seconds. All using scp -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [opensuse] [SLE] Slow transfers from Linux Server
On Wednesday 14 March 2007, Anders Johansson wrote: Just in case you're still interested, I'm doing this now, and I'm getting a constant data rate of over 10MB/s (in real data, not bits over the wire). This means a 100MB file transfers in 9 seconds, or 1024MB in 1 minute 34 seconds. All using scp I don't doubt that Anders, that performance is quite acceptable. My point was, that without testing a samba or nfs transfer you have no way of judging the load imposed by scp. The OP did those tests, and his 35minute transfer with ssh dropped to 5 minutes. He suspects a faulty ssh client, as do I, because that much difference it way out of line with what I would expect. -- _ John Andersen pgp8c30sc7ENc.pgp Description: PGP signature
Re: [opensuse] [SLE] Slow transfers from Linux Server
On Tuesday 13 March 2007, Tim Hempstead wrote: Accessing the system via samba from a Windows XP box seems quite slow as does accessing it via SFTP, (a sustained SFTP transfer using Filezilla peaked at 310kb/s a 670MB iso image has just taken 35+ minutes to transfer across between them). Doesn't sftp require encrypting the file for sending? Samba should outperform sftp. From the Bonnie figures I am guessing the issue is more likely to lie on the networking side rather than the disk side? Ipv6 turned off? You are getting less than 100megabit Cat5 performance. I just copied a 350meg iso across 100mbit network via samba in under 10 minutes. It pegged my linux nic at 7.4 meg for the duration according to gkrellm. So i would put that file on flat disk space (no raid) and copy it with samba to see if the problem is in the disk or the network. You definitely want to get sftp out of the picture. -- _ John Andersen pgptWKTOi5da8.pgp Description: PGP signature
Re: [opensuse] [SLE] Slow transfers from Linux Server
Am Dienstag, den 13.03.2007, 09:50 -0900 schrieb John Andersen: I just copied a 350meg iso across 100mbit network via samba in under 10 minutes. It pegged my linux nic at 7.4 meg for the duration according to gkrellm. I normally see 10M traffic in gkrellm when I copy stuff to and from my nfs server So i would put that file on flat disk space (no raid) and copy it with samba to see if the problem is in the disk or the network. You definitely want to get sftp out of the picture. Two comments to this: first of all, it would have exactly no effect on the data seen in e.g. gkrellm (unless you have very slow cpus), since it measures bits on the wire, not data received by the application Secondly, don't be so quick to discount ssh file transfers. It is heavy on the cpu, but it can even be quicker than plaintext to transfer data if the cpu can keep up. The encryption also does some level of compression, and I haven't been disappointed by the performance so far -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [opensuse] [SLE] Slow transfers from Linux Server
On Tuesday 13 March 2007, Anders Johansson wrote: Am Dienstag, den 13.03.2007, 09:50 -0900 schrieb John Andersen: I just copied a 350meg iso across 100mbit network via samba in under 10 minutes. It pegged my linux nic at 7.4 meg for the duration according to gkrellm. I normally see 10M traffic in gkrellm when I copy stuff to and from my nfs server All the more reason to suspect the OP has network problems. But I have slow disks. Perhaps thats why 7.4 was the fastest my gkrellm showed. Heck, one of my machines was an ancient dual celeron and I still was faster than his reported results. And my switch, while saying 10/100 on the front can not necessarily sustain that packet forwarding rate for long durations. You'd be amazed (or perhaps you wouldn't) how often a supposedly 100meg switch can not actually manage that transfer rate for more than a brief periods. So i would put that file on flat disk space (no raid) and copy it with samba to see if the problem is in the disk or the network. You definitely want to get sftp out of the picture. Two comments to this: first of all, it would have exactly no effect on the data seen in e.g. gkrellm (unless you have very slow cpus), since it measures bits on the wire, not data received by the application Not sure what that has to do with it. I timed this movement by my watch, not gkrellm. Secondly, don't be so quick to discount ssh file transfers. It is heavy on the cpu, but it can even be quicker than plaintext to transfer data if the cpu can keep up. The encryption also does some level of compression, and I haven't been disappointed by the performance so far How much compression would you expect on an iso? Have you tried to move a 650meg iso across nfs, and then do the same move across ssh from and to the same source/destination? I think you will find that on local networks where nothing is less than 100meg that ssh is quite a bit slower than a well tuned nfs. Samba is supposedly not as fast as nfs, but I've found it still is pretty swift compared to ssh transfers. -- _ John Andersen pgpV6gfDMQu1x.pgp Description: PGP signature
Re: [opensuse] [SLE] Slow transfers from Linux Server
On 3/13/07, John Andersen [EMAIL PROTECTED] wrote: I think you will find that on local networks where nothing is less than 100meg that ssh is quite a bit slower than a well tuned nfs. As you said the magic word well tuned nfs ... :) Please, define well-tuned. Or direct me to a very nice tutorial for this. I found a bunch all over the place, and I have very bad experience with the nfs performance while writing to nfs volumes. Very often it stops the transfer for some seconds, konqeror reporting Stall and then resumes. The last part of the file usually takes along time, etc. These observations are made using konqueror, midnight commander, and pure cp from commandline. Sometimes even the overall responsivness of the machine is lost (and this is 3400+ amd with 2G). The files in question are usually more than 300M, and they start pretty well, but after the first 50-70 MB it starts to stall. I found out that using sftp takes about the same amount of time, but does not hog the mouse movement or window switching, as nfs write does. -- Svetoslav Milenov (Sunny) Even the most advanced equipment in the hands of the ignorant is just a pile of scrap. -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [opensuse] [SLE] Slow transfers from Linux Server
On Tue, 2007-03-13 at 16:17 -0500, Sunny wrote: On 3/13/07, John Andersen [EMAIL PROTECTED] wrote: I think you will find that on local networks where nothing is less than 100meg that ssh is quite a bit slower than a well tuned nfs. As you said the magic word well tuned nfs ... :) Please, define well-tuned. Or direct me to a very nice tutorial for this. I found a bunch all over the place, and I have very bad experience with the nfs performance while writing to nfs volumes. Very often it stops the transfer for some seconds, konqeror reporting Stall and then resumes. The last part of the file usually takes along time, etc. These observations are made using konqueror, midnight commander, and pure cp from commandline. Sometimes even the overall responsivness of the machine is lost (and this is 3400+ amd with 2G). The files in question are usually more than 300M, and they start pretty well, but after the first 50-70 MB it starts to stall. I found out that using sftp takes about the same amount of time, but does not hog the mouse movement or window switching, as nfs write does. I would suspect that this has to do with file system caching. The file won't be saved to disk straight away, but to memory. When the memory fills up, it will flush to disk. Does it make a difference if the box has just been rebooted compared to when the box has been up and running for a while? The sftp would be a bit slower generally as it is encrypted, so the target machine will not be hammered, thus being able to flush to disk without you noticing. -- Svetoslav Milenov (Sunny) Cheers, Magnus -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [opensuse] [SLE] Slow transfers from Linux Server
On Tuesday 13 March 2007, Sunny wrote: Please, define well-tuned. Or direct me to a very nice tutorial for this. Oh, no you don't Fella! ;-) I am not an nfs techie. I know very little about it, and only use if for MythTV shares, and I took the parms directly out of the mythtv how-to. So I'm not the guy you would look to for answers. Ask anders, he uses it daily. -- _ John Andersen pgpYuSXuPnkmi.pgp Description: PGP signature
Re: [opensuse] [SLE] Slow transfers from Linux Server
On Tuesday 13 March 2007, Tim Hempstead wrote: John, Ok, I've tested again with both samba and SFTP. Samba is significantly quicker than SFTP with the iso file copying in ~5-6 minutes instead of the 35+ being shown by SFTP. But looking at top whilst the processes are running the system is doing virtually nothing during both transfers, (both smbd and sshd during the respective transfers are peaking at 5% CPU usage, Hmmm, thats more improvement that even I would have expected. Is top showing you the Nice time too? Perhaps the processing has been niced out of the display? But to avoid getting side tracked, does that performance live up to your expectations when running under samba? Can we rule out problems with the hard drive array as well as the network? If so, it sounds like an encryption problem somewhere, and the windows side looks guilty to me. I've never had much luck compressing an ISO, is your sftp trying to use compression in addition to encryption? --- Interesting (and perhaps unrelated) side note regarding ssh: somewhere along the way (in the last month or so) ssh connections started treating the UseDNS yes parameter differently than in the past on one of my servers, either that or bind is horked. The symptom taking was 30 seconds to connect, and from there on running at normal speed. 30 seconds tipped me off to the fact that it was waiting for dns to time out. UseDNS causes it to reverse map the dns to see that it gets something that resolves back to the machine trying to connect. With out host entries on the dns server for local machines it was taking forever.Adding entries to hosts fixed it. Turning off UseDNS would have also been an option. -- _ John Andersen pgpOEiQhIg1T0.pgp Description: PGP signature
Re: [opensuse] [SLE] Slow transfers from Linux Server
Its certainly strange, the samba transfer rate is more the sort of level I was expecting. Top is supposedly showing nice time as well, and running a straight sar instead also gave the same results. Disabling ipv6, UseDNS(*), compression on the SFTP windows client all made little or no difference. Interestingly changing to another sftp client on the wintel end, (the sftp from the putty suite instead of filezilla) appears to run quicker but with occasional large slowdowns with very high CPU usage on the client, (but not the server) ... transfer using this was 12mins, not great but better than before I think that you are correct and that this is a client issue not a problem with the linux server. Cheers Tim (*) although this isn't the problem here I think I may be encountering this elsewhere so cheers for that too :) On 3/13/07, John Andersen [EMAIL PROTECTED] wrote: On Tuesday 13 March 2007, Tim Hempstead wrote: John, Ok, I've tested again with both samba and SFTP. Samba is significantly quicker than SFTP with the iso file copying in ~5-6 minutes instead of the 35+ being shown by SFTP. But looking at top whilst the processes are running the system is doing virtually nothing during both transfers, (both smbd and sshd during the respective transfers are peaking at 5% CPU usage, Hmmm, thats more improvement that even I would have expected. Is top showing you the Nice time too? Perhaps the processing has been niced out of the display? But to avoid getting side tracked, does that performance live up to your expectations when running under samba? Can we rule out problems with the hard drive array as well as the network? If so, it sounds like an encryption problem somewhere, and the windows side looks guilty to me. I've never had much luck compressing an ISO, is your sftp trying to use compression in addition to encryption? --- Interesting (and perhaps unrelated) side note regarding ssh: somewhere along the way (in the last month or so) ssh connections started treating the UseDNS yes parameter differently than in the past on one of my servers, either that or bind is horked. The symptom taking was 30 seconds to connect, and from there on running at normal speed. 30 seconds tipped me off to the fact that it was waiting for dns to time out. UseDNS causes it to reverse map the dns to see that it gets something that resolves back to the machine trying to connect. With out host entries on the dns server for local machines it was taking forever.Adding entries to hosts fixed it. Turning off UseDNS would have also been an option. -- _ John Andersen -- Tim Hempstead [EMAIL PROTECTED] -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
