Re: [CentOS] SFTP - stalled - on large files

2009-12-27 Thread Sean Carolan
 # Turn off SACK
 net.ipv4.tcp_sack = 0

 and execute sysctl -p to apply it.  You can also use sysctl -w
 net.ipv4.tcp_sack=0 to turn it off temporarily.  Our file transfers worked
 just fine after the change.

 I realize there are differences our situation and yours and this might not
 work in your case.  Given the length of this thread, though, it might be
 worth a try!

It appears that the Netscaler load balancer was the problem.  We
turned off TCP buffering (TCPB) on the netscaler for this particular
virtual server, and I was immediately able to transfer a 95MB file
with no issues.  Citrix has acknowledged that there may be some issues
with the tcp stack on this device, which they think have been resolved
in more recent versions of the Netscaler OS.

Hopefully if anyone else experiences this issue, they'll be able to
Google it via the CentOS list archives.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] SFTP - stalled - on large files

2009-12-22 Thread Sean Carolan
 I'm not sure what would cause that, but I'd use rsync over ssh instead of sftp
 anyway - and use the -P option to permit restarting.

If it were up to me, we'd take that route.  The software the client is
using is WinSCP which does have a restart feature, however it's not
working for us.  I'm wondering if this is somehow caused by the vmware
network driver?
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] SFTP - stalled - on large files

2009-12-22 Thread Kai Schaetzl
Sean Carolan wrote on Tue, 22 Dec 2009 03:08:53 -0600:

 The software the client is
 using is WinSCP which does have a restart feature, however it's not
 working for us.

Tell him to switch WinSCP to SCP mode.

Kai

-- 
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com



___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] SFTP - stalled - on large files

2009-12-22 Thread Sean Carolan
 Tell him to switch WinSCP to SCP mode.

 Kai

Tried that, it still fails the same way.  Here's the short list of
what I've tried to troubleshoot this:

Used SCP via the gui and command line
Used SFTP via the gui and command line
Ran yum update to bring all packages up to date
Tried stock CentOS sshd daemon (version 4.3), as well as sshd built
from source (version 5.3)
Adjusted MTU settings
Reinstalled virtual network card
Updated vmware tools and network card driver
Tried vmxnet as well as e1000 drivers

At this point I don't know what else to try.  I'm thinking that it's
either a problem with VMWare, or perhaps our load balancer that is
routing the packets back and forth.  Hopefully one of the vendors will
be able to help solve the problem.  In the meantime we are building
out a physical server to test whether vmware is the issue or not.

If anyone else has seen this problem before or has suggestions please
post them here.  Thanks.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] SFTP - stalled - on large files

2009-12-22 Thread JS


 -Original Message-
 From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On
 Behalf Of Sean Carolan
 Sent: Tuesday, December 22, 2009 6:13 AM
 To: CentOS mailing list
 Subject: Re: [CentOS] SFTP - stalled - on large files
 
  Tell him to switch WinSCP to SCP mode.
 
  Kai
 
 Tried that, it still fails the same way.  Here's the short list of
 what I've tried to troubleshoot this:
 
 Used SCP via the gui and command line
 Used SFTP via the gui and command line
 Ran yum update to bring all packages up to date
 Tried stock CentOS sshd daemon (version 4.3), as well as sshd built
 from source (version 5.3)
 Adjusted MTU settings
 Reinstalled virtual network card
 Updated vmware tools and network card driver
 Tried vmxnet as well as e1000 drivers
 
 At this point I don't know what else to try.  I'm thinking that it's
 either a problem with VMWare, or perhaps our load balancer that is
 routing the packets back and forth.  Hopefully one of the vendors will
 be able to help solve the problem.  In the meantime we are building
 out a physical server to test whether vmware is the issue or not.
 
 If anyone else has seen this problem before or has suggestions please
 post them here.  Thanks.
---

Just an idea or thought on it.  You never said what the file size was or did
you?  My idea is that is, there not a file size limitation on transfer to
and from the server?  I thought there was?  Check you vsftpd.conf out or
what ever ftp server your running for the size limitation.  Maybe some help
or maybe not?

John 

___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] SFTP - stalled - on large files

2009-12-22 Thread Sean Carolan
 Just an idea or thought on it.  You never said what the file size was or did
 you?  My idea is that is, there not a file size limitation on transfer to
 and from the server?  I thought there was?  Check you vsftpd.conf out or
 what ever ftp server your running for the size limitation.  Maybe some help
 or maybe not?

The problem is with SFTP, so I'm afraid that vsftpd.conf isn't the culprit here.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] SFTP - stalled - on large files

2009-12-22 Thread Sean Carolan
 Load balancer... is that set up to maintain connections, or will it, like 
 IBM's
 WebSeal, go to whichever server is next/least used in the middle of a 
 connection?

It's set to use least connection but there is only one server behind
the virtual IP at the moment.

I'm reasonably sure at this point that the Netscaler is causing the
problem, because file transfers inside the LAN work fine, and we see
this same issue on both physical and virtual servers.  I just tested
with a physical box to verify, and the same thing happens, transfer
speed quickly drops to zero and stalls.

I've got a ticket open with Citrix to hopefully get to the bottom of
this.  It wouldn't be the first time we've seen the Netscaler muck up
a TCP connection from a client.  The last time I dealt with this it
was sending unwanted FIN packets to mail servers.  Fun stuff.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] SFTP - stalled - on large files

2009-12-22 Thread Jay Leafey
We had a similar problem copying files between servers on two of our 
campuses via SCP.  After a while the connection just stalled out and 
hung.  The problem turned out to be SCP and SFTP interacting a bug in 
the SACK (Selective Acknowledgment) algorithm used in Linux.  We turned 
it off on the two endpoints using the following addition to 
/etc/sysctl.conf:



# Turn off SACK
net.ipv4.tcp_sack = 0


and execute sysctl -p to apply it.  You can also use sysctl -w 
net.ipv4.tcp_sack=0 to turn it off temporarily.  Our file transfers 
worked just fine after the change.


I realize there are differences our situation and yours and this might 
not work in your case.  Given the length of this thread, though, it 
might be worth a try!

--
Jay Leafey - Memphis, TN
jay.lea...@mindless.com


smime.p7s
Description: S/MIME Cryptographic Signature
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos


[CentOS] SFTP - stalled - on large files

2009-12-21 Thread Sean Carolan
I have an SSH server that was set up for a client, and every time we
try to upload large files via SFTP or scp, the transfers speed quickly
slows to zero and gives a - stalled - status message, then
disconnects.  Here is an example:

ftp put iTunesSetup.exe iTunesSetup.exe
Uploading iTunesSetup.exe to /home/scarolan/iTunesSetup.exe
iTunesSetup.exe 0%  704KB   0.0KB/s -
stalled -debug1: channel 0: free: client-session, nchannels 1
Read from remote host ftp.authoria.net: Connection reset by peer
debug1: Transferred: stdin 0, stdout 0, stderr 66 bytes in 203.9 seconds
debug1: Bytes per second: stdin 0.0, stdout 0.0, stderr 0.3
debug1: Exit status -1
Connection closed

I have no idea why this is happening.  Can anyone point me in the
right direction?
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] SFTP - stalled - on large files

2009-12-21 Thread Sean Carolan
On Mon, Dec 21, 2009 at 7:06 PM, 唐建伟 myh...@gmail.com wrote:

  I met the same as you, but always due to the bad network connection.


I should probably provide some more information, the server is a VMware
guest running CentOS 5.3.  It's using the vmxnet driver for the eth0
connection.  IPv6 is disabled.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos


Re: [CentOS] SFTP - stalled - on large files

2009-12-21 Thread Les Mikesell
Sean Carolan wrote:
 On Mon, Dec 21, 2009 at 7:06 PM, 唐建伟 myh...@gmail.com 
 mailto:myh...@gmail.com wrote:
 
  I met the same as you, but always due to the bad network connection.
 
 
 I should probably provide some more information, the server is a VMware 
 guest running CentOS 5.3.  It's using the vmxnet driver for the eth0 
 connection.  IPv6 is disabled. 

I'm not sure what would cause that, but I'd use rsync over ssh instead of sftp 
anyway - and use the -P option to permit restarting.

-- 
   Les Mikesell
lesmikes...@gmail.com
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos