Thanks Harold. Will check out the patch.
The scp "problem" w/ transfer rates that don't go above 8 MB/sec turns out
to be a CPU bottleneck. Our z9-109 doesn't have any crypto hardware so all
the encryption/decryption is being done in software. My speculation about
there being an I/O bottleneck due to scp not using cached files is
incorrect - caching is used, but the CPU approaches 50% for each of the two
virtual servers involved (sender and receiver), which kinda swamps my
single IFL engine. %-)
Best regards,
Mark
Harold Grovesteen
<[EMAIL PROTECTED]
r.com> To
Sent by: Linux on LINUX-390@VM.MARIST.EDU
390 Port cc
<[EMAIL PROTECTED]
IST.EDU> Subject
Re: Hipersocket Performance Problem
on SLES 9
01/19/2007 04:55
AM
Please respond to
Linux on 390 Port
<[EMAIL PROTECTED]
IST.EDU>
From James Melin's 12/14/2006 post:
Mark, I saw something similar when we first went to SLES-9 (64 bit). We
had MTU sizes being negotiated down to an idiotic packet size of 1492.
Whether or not this has to do with how our OSA is configured I never did
learn. Search for stuff with hipersockets and MTU in the archives for the
particulars.
We reported it to Novell. They eventually managed to test on SLES 9 under
z/Series and eventually issued a patch. They sent us this file:
iputils-ss021109-147.2.PTF.184411.0.s390.rpm
I don't know whether or not this file is GA but it touched MANY IP related
items, indicating that the problem was pervasive. Once we applied it our
MTU negotiation problem went away. It may or may not have bearing on your
situation, but I offer the information to any and all that might find it
relevant.
-J
barton wrote:
> I've seen this a lot. You need to find the real bottleneck. Are
> you equiped?
>
>
>
> Mark Wheeler wrote:
>
>> More info:
>>
>> FTP GET's of this cached 256MB file to /dev/null have run as fast as 205
>> MB/sec w/ MTU=32760 over hipersocket interfaces, and 168 MB over virtual
>> CTCs. FTP PUT's are a different matter: hipersocket or CTC doesn't
>> make a
>> difference. The higher MTU size, the slower the transfer rate (from
>> 90-100
>> MB/sec w/ MTU=16376 down to 10-15 MB/sec for MTU=32084. At MTU=32085,
>> performance drops off a cliff, down to 400 KB/sec.
>>
>> So, one problem appears to be related to FTP PUT performance with
>> large MTU
>> sizes.
>>
>> In the process of investigating this alleged "hipersocket performance
>> problem" (as reported by our Linux support person up four chains of
>> management), I discovered an interesting issue with scp. It shows very
>> limitted sensitivity for interface type, MTU size, or direction of
>> movement), however I can't get it to run faster than 8 MB/sec (25 times
>> slower than FTP GET). I'm using scp between two virtual Linux machines
>> running on a 1-IFL system, so thought initially it may have been CPU
>> contention (no hardware crypto on this box), yet CPU util was less
>> than 5%
>> on each machine. What I did see was the sending side doing a lot of disk
>> I/O, something I don't see with FTP. Is it possible that scp reads from
>> disk, bypassing the cache?
>>
>> Linux version 2.6.5-7.276-s390x
>>
>> Best regards,
>> Mark
>>
>>
>>> Greetings all,
>>>
>>> I've been using real hipersockets for a couple years now. Recently a
>>> significant performance problem with SLES 9 and large MTU sizes was
>>
>>
>> brought
>>
>>> to my attention. I've been using MTU=32760. I set up a test to
>>> illustrate
>>> the problem. I built a 256 MB file on one zLinux guest (running under
>>
>>
>> z/VM
>>
>>> 5.2 on a z9-109), with enough storage defined so the file could be
>>> completely cached. I then FTP'd it to /dev/null on another server
>>> over a
>>> hipersocket connection. Here's what I observed:
>>> SLES8-to-SLES8, MTU=8184, ~75 MB/sec
>>> SLES8-to-SLES8, MTU=32760, ~100 MB/sec (as high as 132 MB/sec)
>>> SLES8-to-SLES9, MTU=32760, ~100 MB/sec
>>> z/VM-to-SLES9, MTU=32760, ~100 MB/sec (from file on VDISK to
>>> /dev/null)
>>> z/OS-to-SLES9, MTU=32760, ~25 MB/sec (from disk cache on z/OS)
>>> SLES9-to-SLES9, MTU=8184, ~75 MB/sec
>>> SLES9-to-SLES9, MTU=32760, ~400 KB/sec (not MegaBytes, KiloBytes!)
>>>
>>> Has anyone else seen this?
>>>
>>> I use 32760 on my hipersocket links so as to be consistent with the
>>> 32760
>>> MTU size used on CTC links to other processors owned by my z/VM TCPIP
>>> machine, which is used as the gateway between Linux guests on one
>>> machine
>>> and z/OS on the other machines.
>>>
>>> Best regards,
>>> Mark Wheeler, 3M Company
>>>
>>> ----------------------------------------------------------------------
>>> For LINUX-390 subscribe / signoff / archive access instructions,
>>> send email to [EMAIL PROTECTED] with the message: INFO
>>> LINUX-390 or
>>
>>
>> visit
>>
>>> http://www.marist.edu/htbin/wlvindex?LINUX-390
>>>
>>> ----------------------------------------------------------------------
>>> For LINUX-390 subscribe / signoff / archive access instructions,
>>> send email to [EMAIL PROTECTED] with the message: INFO
>>> LINUX-390 or
>>
>>
>> visit
>>
>>> http://www.marist.edu/htbin/wlvindex?LINUX-390
>>>
>>
>>
>> ----------------------------------------------------------------------
>> For LINUX-390 subscribe / signoff / archive access instructions,
>> send email to [EMAIL PROTECTED] with the message: INFO LINUX-390
>> or visit
>> http://www.marist.edu/htbin/wlvindex?LINUX-390
>>
>>
>>
>
>
> ----------------------------------------------------------------------
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: INFO LINUX-390
> or visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
----------------------------------------------------------------------
For LINUX-390 subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
----------------------------------------------------------------------
For LINUX-390 subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
