QETH issue?
On an occasional basis, when I have to IPL a guest under 5.4, I find that the primary network connection doesn't work properly when the SLES system comes [back] up. It isn't until I ping the gateway from said guest that I'm able to access it remotely. Until that step is performed, the system is inaccessible over the network. I accomplish this either by accessing the guest from another systems over a Hipersockets connection or the TN3270 console. After this step is performed, it's again possible to access the guest remotely, via SSH, over the primary network connection. All of the SLES guests are attached to a VLAN-aware VSWITCH, which in turn is connected to redundant OSA Express-3 cards. Has anyone else seen this behavior? Any idea why it might do this? It doesn't appear to matter whether it's a SLES 10 or SLES 11 guest, the issue is the same. It is NOT every time the guests is IPL'd, either which makes it all the more bizzare and difficult to troubleshoot. Thanks, Dave Dave Keeton ETS / Service Delivery State Data Center (503) 373-0832 Confidentiality Note: This electronic mail transmission contains information belonging to the Department of Administrative Services, Enterprise Technical Services. This information may be confidential and/or legally privileged and is intended for the use of the addressees designated above. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or the taking of any action in reliance on the contents of this electronic information is prohibited. If you have received this transmission in error, please delete and notify us immediately. -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: QETH issue?
Yes, I have seen this on every sles guest I installed on a client. Seems like sles don't do an arp request or arp announce (I don't remember the arp direction now) when it puts the interface online. I put there a hack that solved. After all initialization scripts I sent a ping -c 3 to the gateway. And it works. On Aug 14, 2012 12:30 PM, KEETON Dave * ETS dave.kee...@state.or.us wrote: On an occasional basis, when I have to IPL a guest under 5.4, I find that the primary network connection doesn't work properly when the SLES system comes [back] up. It isn't until I ping the gateway from said guest that I'm able to access it remotely. Until that step is performed, the system is inaccessible over the network. I accomplish this either by accessing the guest from another systems over a Hipersockets connection or the TN3270 console. After this step is performed, it's again possible to access the guest remotely, via SSH, over the primary network connection. All of the SLES guests are attached to a VLAN-aware VSWITCH, which in turn is connected to redundant OSA Express-3 cards. Has anyone else seen this behavior? Any idea why it might do this? It doesn't appear to matter whether it's a SLES 10 or SLES 11 guest, the issue is the same. It is NOT every time the guests is IPL'd, either which makes it all the more bizzare and difficult to troubleshoot. Thanks, Dave Dave Keeton ETS / Service Delivery State Data Center (503) 373-0832 Confidentiality Note: This electronic mail transmission contains information belonging to the Department of Administrative Services, Enterprise Technical Services. This information may be confidential and/or legally privileged and is intended for the use of the addressees designated above. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or the taking of any action in reliance on the contents of this electronic information is prohibited. If you have received this transmission in error, please delete and notify us immediately. -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/ -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: QETH issue?
Hi Dave. Mauro had a workaround for such issue, described here: http://www.mail-archive.com/linux-390@vm.marist.edu/msg61817.html He did not state which was the script, but I'll assume you could just put a 'ping' test into 'rc.local', so it pings the gateway when it reaches that step. HTH, Pedro Principeza From: KEETON Dave * ETS dave.kee...@state.or.us To: LINUX-390@vm.marist.edu, Date: 14/08/2012 12:28 Subject:QETH issue? Sent by:Linux on 390 Port LINUX-390@vm.marist.edu On an occasional basis, when I have to IPL a guest under 5.4, I find that the primary network connection doesn't work properly when the SLES system comes [back] up. It isn't until I ping the gateway from said guest that I'm able to access it remotely. Until that step is performed, the system is inaccessible over the network. I accomplish this either by accessing the guest from another systems over a Hipersockets connection or the TN3270 console. After this step is performed, it's again possible to access the guest remotely, via SSH, over the primary network connection. All of the SLES guests are attached to a VLAN-aware VSWITCH, which in turn is connected to redundant OSA Express-3 cards. Has anyone else seen this behavior? Any idea why it might do this? It doesn't appear to matter whether it's a SLES 10 or SLES 11 guest, the issue is the same. It is NOT every time the guests is IPL'd, either which makes it all the more bizzare and difficult to troubleshoot. Thanks, Dave Dave Keeton ETS / Service Delivery State Data Center (503) 373-0832 Confidentiality Note: This electronic mail transmission contains information belonging to the Department of Administrative Services, Enterprise Technical Services. This information may be confidential and/or legally privileged and is intended for the use of the addressees designated above. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or the taking of any action in reliance on the contents of this electronic information is prohibited. If you have received this transmission in error, please delete and notify us immediately. -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/ -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Fw: QETH issue?
My bad! It is described here: http://www.mail-archive.com/linux-390@vm.marist.edu/msg61824.html - Forwarded by Pedro Henrique dos Santos Principeza/Brazil/IBM on 14/08/2012 12:37 - From: Pedro Henrique dos Santos Principeza/Brazil/IBM To: Linux on 390 Port LINUX-390@vm.marist.edu, Date: 14/08/2012 12:33 Subject:Re: QETH issue? Hi Dave. Mauro had a workaround for such issue, described here: http://www.mail-archive.com/linux-390@vm.marist.edu/msg61817.html He did not state which was the script, but I'll assume you could just put a 'ping' test into 'rc.local', so it pings the gateway when it reaches that step. HTH, Pedro Principeza From: KEETON Dave * ETS dave.kee...@state.or.us To: LINUX-390@vm.marist.edu, Date: 14/08/2012 12:28 Subject:QETH issue? Sent by:Linux on 390 Port LINUX-390@vm.marist.edu On an occasional basis, when I have to IPL a guest under 5.4, I find that the primary network connection doesn't work properly when the SLES system comes [back] up. It isn't until I ping the gateway from said guest that I'm able to access it remotely. Until that step is performed, the system is inaccessible over the network. I accomplish this either by accessing the guest from another systems over a Hipersockets connection or the TN3270 console. After this step is performed, it's again possible to access the guest remotely, via SSH, over the primary network connection. All of the SLES guests are attached to a VLAN-aware VSWITCH, which in turn is connected to redundant OSA Express-3 cards. Has anyone else seen this behavior? Any idea why it might do this? It doesn't appear to matter whether it's a SLES 10 or SLES 11 guest, the issue is the same. It is NOT every time the guests is IPL'd, either which makes it all the more bizzare and difficult to troubleshoot. Thanks, Dave Dave Keeton ETS / Service Delivery State Data Center (503) 373-0832 Confidentiality Note: This electronic mail transmission contains information belonging to the Department of Administrative Services, Enterprise Technical Services. This information may be confidential and/or legally privileged and is intended for the use of the addressees designated above. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or the taking of any action in reliance on the contents of this electronic information is prohibited. If you have received this transmission in error, please delete and notify us immediately. -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/ -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: QETH issue?
Hi, I am sure that i had the exact same issue about 6 years ago (SLES as well older z/VM)... i can't remember for sure what fixed it... I think this was a SRM STORBUF issue... a ping from hiper socket would have caused the linux to get back into the dispatch list from the eligable list. make sure you correctly set the SRM parameters (you can see the machine current queue with PERFSVM while the guest is not responding to ssh from LAN). one more thing that i vaguely remember is something about defined chpids with no devices attached to them... but i am not sure it is related to this issue... I hope this helps... Offer Baruch On Tue, Aug 14, 2012 at 6:26 PM, KEETON Dave * ETS dave.kee...@state.or.uswrote: On an occasional basis, when I have to IPL a guest under 5.4, I find that the primary network connection doesn't work properly when the SLES system comes [back] up. It isn't until I ping the gateway from said guest that I'm able to access it remotely. Until that step is performed, the system is inaccessible over the network. I accomplish this either by accessing the guest from another systems over a Hipersockets connection or the TN3270 console. After this step is performed, it's again possible to access the guest remotely, via SSH, over the primary network connection. All of the SLES guests are attached to a VLAN-aware VSWITCH, which in turn is connected to redundant OSA Express-3 cards. Has anyone else seen this behavior? Any idea why it might do this? It doesn't appear to matter whether it's a SLES 10 or SLES 11 guest, the issue is the same. It is NOT every time the guests is IPL'd, either which makes it all the more bizzare and difficult to troubleshoot. Thanks, Dave Dave Keeton ETS / Service Delivery State Data Center (503) 373-0832 Confidentiality Note: This electronic mail transmission contains information belonging to the Department of Administrative Services, Enterprise Technical Services. This information may be confidential and/or legally privileged and is intended for the use of the addressees designated above. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or the taking of any action in reliance on the contents of this electronic information is prohibited. If you have received this transmission in error, please delete and notify us immediately. -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/ -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: QETH issue?
On Tue, 2012-08-14 at 15:26 +, KEETON Dave * ETS wrote: On an occasional basis, when I have to IPL a guest under 5.4, I find that the primary network connection doesn't work properly when the SLES system comes [back] up. It isn't until I ping the gateway from said guest that I'm able to access it remotely. Until that step is performed, the system is inaccessible over the network. I accomplish this either by accessing the guest from another systems over a Hipersockets connection or the TN3270 console. After this step is performed, it's again possible to access the guest remotely, via SSH, over the primary network connection. All of the SLES guests are attached to a VLAN-aware VSWITCH, which in turn is connected to redundant OSA Express-3 cards. Has anyone else seen this behavior? Any idea why it might do this? It doesn't appear to matter whether it's a SLES 10 or SLES 11 guest, the issue is the same. It is NOT every time the guests is IPL'd, either which makes it all the more bizzare and difficult to troubleshoot. Dave, we reported this problem for SLES 11 in SUSE bugzilla 617373. Solution has been to introduce a SEND_GRATUITOUS_ARP config option. Its default is no. Changing it to yes should trigger the sending of gratuitous ARPs. It applies to ETHERNET (not IP) VSWITCHes. The config option is offered starting with sysconfig-0.71.30-0.10.1. Regards, Ursula Braun, IBM Germany -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: QETH issue?
Even though we don't seem to have this problem, curiosity got me so I looked it up. The sysconfig package has this in its change log: * Mon Apr 18 2011 m...@suse.de - Send gratuitous arp when new SEND_GRATUITOUS_ARP variable is set to yes either in global network/config or in per-interface ifcfg file. Fixed to use CHECK_DUPLICATE_IP for ipv4 only (bnc#617373). To set this, edit /etc/sysconfig/network/config And put YES in ## Type:yesno ## Default: no # If ifup should send a gratuitous ARP to inform the receivers about its # static IP addresses and perhaps also a link-layer (MAC) address change. # Make sure that packet sockets (CONFIG_PACKET) are supported in the kernel, # since this feature uses arping, which depends on that. SEND_GRATUITOUS_ARP=no Marcy This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. -Original Message- From: Linux on 390 Port [mailto:LINUX-390@VM.MARIST.EDU] On Behalf Of Ursula Braun Sent: Tuesday, August 14, 2012 8:55 AM To: LINUX-390@VM.MARIST.EDU Subject: Re: [LINUX-390] QETH issue? On Tue, 2012-08-14 at 15:26 +, KEETON Dave * ETS wrote: On an occasional basis, when I have to IPL a guest under 5.4, I find that the primary network connection doesn't work properly when the SLES system comes [back] up. It isn't until I ping the gateway from said guest that I'm able to access it remotely. Until that step is performed, the system is inaccessible over the network. I accomplish this either by accessing the guest from another systems over a Hipersockets connection or the TN3270 console. After this step is performed, it's again possible to access the guest remotely, via SSH, over the primary network connection. All of the SLES guests are attached to a VLAN-aware VSWITCH, which in turn is connected to redundant OSA Express-3 cards. Has anyone else seen this behavior? Any idea why it might do this? It doesn't appear to matter whether it's a SLES 10 or SLES 11 guest, the issue is the same. It is NOT every time the guests is IPL'd, either which makes it all the more bizzare and difficult to troubleshoot. Dave, we reported this problem for SLES 11 in SUSE bugzilla 617373. Solution has been to introduce a SEND_GRATUITOUS_ARP config option. Its default is no. Changing it to yes should trigger the sending of gratuitous ARPs. It applies to ETHERNET (not IP) VSWITCHes. The config option is offered starting with sysconfig-0.71.30-0.10.1. Regards, Ursula Braun, IBM Germany -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: QETH issue?
On Tuesday, 08/14/2012 at 01:35 EDT, KEETON Dave * ETS dave.kee...@state.or.us wrote: Thank you everyone for the responses. I've been living with this issue for a while; it's been more of an annoyance than anything, so I'm glad I finally got around to asking the question. Many of us are, I think, mystified as to why Linux would stop doing the one thing that gets the attention of an adjacent switch. In abstract, it represents Linux's desire to receive data for this IP address. Switches use the contents of inbound frames (e.g. grat ARP) to learn where to send unicast frames. If the host moves and doesn't say anything, it won't receive any unicast frames until the Dynamic Forwarding Entry in the port forwarding database expires (typ. 5 min), forcing the switch to forward the frame to all ports in that same LAN. In the world of physical switches, a reconfiguration event (plug/enable) causes the switch to briefly lower the expiry interval to force packets down a new quiet port in order to induce the gopher to come out of his burrow, so to speak. But since the switch doesn't get the reconfiguration event, it must rely on Good Behavior from the hosts. Good news is that the Virtual Switch in z/VM continues to evolve, learning how to deal with these issues. Alan Altmark Senior Managing z/VM and Linux Consultant IBM System Lab Services and Training ibm.com/systems/services/labservices office: 607.429.3323 mobile; 607.321.7556 alan_altm...@us.ibm.com IBM Endicott -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: QETH issue?
Thank you everyone for the responses. I've been living with this issue for a while; it's been more of an annoyance than anything, so I'm glad I finally got around to asking the question. -Original Message- From: Linux on 390 Port [mailto:LINUX-390@VM.MARIST.EDU] On Behalf Of Marcy Cortes Sent: Tuesday, August 14, 2012 9:56 AM To: LINUX-390@VM.MARIST.EDU Subject: Re: QETH issue? Even though we don't seem to have this problem, curiosity got me so I looked it up. The sysconfig package has this in its change log: * Mon Apr 18 2011 m...@suse.de - Send gratuitous arp when new SEND_GRATUITOUS_ARP variable is set to yes either in global network/config or in per-interface ifcfg file. Fixed to use CHECK_DUPLICATE_IP for ipv4 only (bnc#617373). To set this, edit /etc/sysconfig/network/config And put YES in ## Type:yesno ## Default: no # If ifup should send a gratuitous ARP to inform the receivers about its # static IP addresses and perhaps also a link-layer (MAC) address change. # Make sure that packet sockets (CONFIG_PACKET) are supported in the kernel, # since this feature uses arping, which depends on that. SEND_GRATUITOUS_ARP=no Marcy This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. -Original Message- From: Linux on 390 Port [mailto:LINUX-390@VM.MARIST.EDU] On Behalf Of Ursula Braun Sent: Tuesday, August 14, 2012 8:55 AM To: LINUX-390@VM.MARIST.EDU Subject: Re: [LINUX-390] QETH issue? On Tue, 2012-08-14 at 15:26 +, KEETON Dave * ETS wrote: On an occasional basis, when I have to IPL a guest under 5.4, I find that the primary network connection doesn't work properly when the SLES system comes [back] up. It isn't until I ping the gateway from said guest that I'm able to access it remotely. Until that step is performed, the system is inaccessible over the network. I accomplish this either by accessing the guest from another systems over a Hipersockets connection or the TN3270 console. After this step is performed, it's again possible to access the guest remotely, via SSH, over the primary network connection. All of the SLES guests are attached to a VLAN-aware VSWITCH, which in turn is connected to redundant OSA Express-3 cards. Has anyone else seen this behavior? Any idea why it might do this? It doesn't appear to matter whether it's a SLES 10 or SLES 11 guest, the issue is the same. It is NOT every time the guests is IPL'd, either which makes it all the more bizzare and difficult to troubleshoot. Dave, we reported this problem for SLES 11 in SUSE bugzilla 617373. Solution has been to introduce a SEND_GRATUITOUS_ARP config option. Its default is no. Changing it to yes should trigger the sending of gratuitous ARPs. It applies to ETHERNET (not IP) VSWITCHes. The config option is offered starting with sysconfig-0.71.30-0.10.1. Regards, Ursula Braun, IBM Germany -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: QETH issue?
* Mon Apr 18 2011 m...@suse.de - Send gratuitous arp when new SEND_GRATUITOUS_ARP variable is set to yes either in global network/config or in per-interface ifcfg file. Fixed to use CHECK_DUPLICATE_IP for ipv4 only (bnc#617373). To set this, edit /etc/sysconfig/network/config And put YES in ## Type:yesno ## Default: no That just seems like a wrong default on this platform, or at least harmless to default to yes on all platforms. Open a bug with SuSE/RH to get this fixed?
Re: QETH issue?
A couple of folks must have switches that are acting differently, perhaps because of the settings Alan was mentioning? Seems like the vast majority don't have this problem. Marcy This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. -Original Message- From: Linux on 390 Port [mailto:LINUX-390@VM.MARIST.EDU] On Behalf Of David Boyes Sent: Tuesday, August 14, 2012 2:24 PM To: LINUX-390@VM.MARIST.EDU Subject: Re: [LINUX-390] QETH issue? * Mon Apr 18 2011 m...@suse.de - Send gratuitous arp when new SEND_GRATUITOUS_ARP variable is set to yes either in global network/config or in per-interface ifcfg file. Fixed to use CHECK_DUPLICATE_IP for ipv4 only (bnc#617373). To set this, edit /etc/sysconfig/network/config And put YES in ## Type:yesno ## Default: no That just seems like a wrong default on this platform, or at least harmless to default to yes on all platforms. Open a bug with SuSE/RH to get this fixed?
Re: QETH issue?
A couple of folks must have switches that are acting differently, perhaps because of the settings Alan was mentioning? Seems like the vast majority don't have this problem. More likely different traffic patterns, or different settings for CAM table expiration in the hardware switches outside the VSWITCH environment. If there were immediate activity on the newly connected port, the vswitch could learn the new MAC addresses quickly enough to avoid the problem. If the newly connected port sits for a long (in ms) time, the switch won't update the forwarding table until the CAM timer expires, and you get the symptom. Most other platforms use the gratuitous ARP at link-state change to force the issue -- to make it reliable, the ARP triggers actual traffic to force the switch to see the new MAC. I don't see why that's a bad thing here - it should be happening here as the default behavior, especially on CSMA hardware like Ethernet. If this were token ring, it wouldn't matter so much, but insertion of a new station into a CSMA broadcast domain should force the update. The current setting breaks duplicate IP address detection. Is one frame too high a price for reliable behavior?
