Re: [WIRELESS-LAN] Internet Connectivity Issues
Hi Sean, Agree with your sentiment, as just as another, we heard recently that .125 would be the last MR in the 8.8 train, forcing customers over to 8.10 for mainline which has not even been released yet… It’s taken a year for 8.8 to get stable enough for a relatively stress-free life, and just as I type this our 3504’s have started flipping between HA pairs for another eventful afternoon. C From: The EDUCAUSE Wireless Issues Community Group Listserv on behalf of "Gray, Sean" Reply-To: The EDUCAUSE Wireless Issues Community Group Listserv Date: Tuesday, 24 September 2019 at 15:59 To: Subject: Re: [WIRELESS-LAN] Internet Connectivity Issues Hi Carlo, We will be upgrading to the TAC recommended 8.8.125.0 later tonight. We thought that this could eliminate some problems and at least squash a few bugs that we may be running into. I really wish Cisco would make an MD release available for the 8.8 train, it seems to have been a continual ED train since it came out of the gate. Thanks Sean Sean Gray | B.Sc (Hons) Voice, Collaboration & Wireless Network Analyst ITS, University of Lethbridge From: The EDUCAUSE Wireless Issues Community Group Listserv On Behalf Of Carlo Terminiello Sent: September 24, 2019 7:41 AM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU Subject: Re: [WIRELESS-LAN] Internet Connectivity Issues Hi, We were hit with similar problems across multiple sites on earlier 8.8 builds, now running the latest/ last 8.8.125.3 escalation build from Cisco on multiple controllers with stable results. We are seeing one issue with MBP whereby they remain associated and on session timeout stop forwarding traffic. This is a corner case and yet to be successfully identified as a bug by Cisco TAC. Rgds Carlo On Tue, 24 Sep 2019 at 14:18, John Turner wrote: I second Mark's suggestion - we've detected this problem at nearly a dozen of our higher ed and enterprise customers. Seems that the issue is between a waking macbook and an android https://www.cisco.com/c/en/us/support/docs/wireless/aironet-3800-series-access-points/214491-arp-responses-for-default-gateway-ip-add.html Traditional tools aren't finding this - we are detecting a drop in traffic as well as the duplicate gateway address that triggers an advisory alert. John On Tue, Sep 24, 2019 at 8:57 AM Mark Reboli wrote: Look into the Issue with apple sleeping devices changing their IPs to your gateway. We experienced this causing intermitted issues. Take a look at the link below: https://community.meraki.com/t5/Wireless-LAN/Help-Windows-client-receiving-incorrect-MAC-address-of-Gateway/td-p/39963 If you have any questions please feel free to contact me off line m Mark Reboli Network/Telecom Manager Misericordia University (570) 674-6753 This e-mail and accompanying attachments are confidential. The information is intended solely for the use of the individual to whom it is addressed. Any review, disclosure, copying, distribution, or use of this e-mail communication by others is strictly prohibited. If you are not the intended recipient, please notify us immediately by returning this message to the sender and delete all copies. Thank you for your cooperation. From: The EDUCAUSE Wireless Issues Community Group Listserv [mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU] On Behalf Of Ethan Grinnell Sent: Monday, September 23, 2019 11:04 PM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU Subject: Re: [WIRELESS-LAN] Internet Connectivity Issues External Email: Do not click any links or open any attachments unless you trust the sender and know the content is safe. What AP models do you have? We're still on AireOS 8.5.144.33. The COS APs have been running pretty well for us. We've had intermittent issues on IOS APs though. The radios periodically reset and can't seem to clear some packets. Once an AP starts having issues the radio may even cycle every few minutes indefinitely. Do you monitor the AP syslogs? Here are the bug IDs for this: CSCvp86151 IOS APs radio reset with code 44, mostly seen on 2.4GHz radio CSCvp66546 702w Radio reset due to tx stuck The first one doesn't have a real solution yet, we're running debug images to capture more information. TAC says it might be related to high QBSS, but I saw an AP report high QBSS with no other APs or other known Wi-Fi sources above -80 dBm, so I'm not convinced. The 702W bug has a good debug image in place though. TAC found that stranded packets for clients in power save mode (mostly Murata or Samsung S10 devices) would accumulate in memory and after 60 seconds the AP radio resets to clear the packets. But the reset didn't clear them (or more accumulated) so it just continues resetting the radio. The fixed image discards packets that couldn't be transmitted after 20 second
RE: [WIRELESS-LAN] Internet Connectivity Issues
Hi Carlo, We will be upgrading to the TAC recommended 8.8.125.0 later tonight. We thought that this could eliminate some problems and at least squash a few bugs that we may be running into. I really wish Cisco would make an MD release available for the 8.8 train, it seems to have been a continual ED train since it came out of the gate. Thanks Sean Sean Gray | B.Sc (Hons) Voice, Collaboration & Wireless Network Analyst ITS, University of Lethbridge From: The EDUCAUSE Wireless Issues Community Group Listserv On Behalf Of Carlo Terminiello Sent: September 24, 2019 7:41 AM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU Subject: Re: [WIRELESS-LAN] Internet Connectivity Issues Hi, We were hit with similar problems across multiple sites on earlier 8.8 builds, now running the latest/ last 8.8.125.3 escalation build from Cisco on multiple controllers with stable results. We are seeing one issue with MBP whereby they remain associated and on session timeout stop forwarding traffic. This is a corner case and yet to be successfully identified as a bug by Cisco TAC. Rgds Carlo On Tue, 24 Sep 2019 at 14:18, John Turner mailto:jtur...@nyansa.com>> wrote: I second Mark's suggestion - we've detected this problem at nearly a dozen of our higher ed and enterprise customers. Seems that the issue is between a waking macbook and an android https://www.cisco.com/c/en/us/support/docs/wireless/aironet-3800-series-access-points/214491-arp-responses-for-default-gateway-ip-add.html Traditional tools aren't finding this - we are detecting a drop in traffic as well as the duplicate gateway address that triggers an advisory alert. John On Tue, Sep 24, 2019 at 8:57 AM Mark Reboli mailto:mreb...@misericordia.edu>> wrote: Look into the Issue with apple sleeping devices changing their IPs to your gateway. We experienced this causing intermitted issues. Take a look at the link below: https://community.meraki.com/t5/Wireless-LAN/Help-Windows-client-receiving-incorrect-MAC-address-of-Gateway/td-p/39963 If you have any questions please feel free to contact me off line m Mark Reboli Network/Telecom Manager Misericordia University (570) 674-6753 This e-mail and accompanying attachments are confidential. The information is intended solely for the use of the individual to whom it is addressed. Any review, disclosure, copying, distribution, or use of this e-mail communication by others is strictly prohibited. If you are not the intended recipient, please notify us immediately by returning this message to the sender and delete all copies. Thank you for your cooperation. From: The EDUCAUSE Wireless Issues Community Group Listserv [mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU<mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU>] On Behalf Of Ethan Grinnell Sent: Monday, September 23, 2019 11:04 PM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU<mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU> Subject: Re: [WIRELESS-LAN] Internet Connectivity Issues External Email: Do not click any links or open any attachments unless you trust the sender and know the content is safe. What AP models do you have? We're still on AireOS 8.5.144.33. The COS APs have been running pretty well for us. We've had intermittent issues on IOS APs though. The radios periodically reset and can't seem to clear some packets. Once an AP starts having issues the radio may even cycle every few minutes indefinitely. Do you monitor the AP syslogs? Here are the bug IDs for this: CSCvp86151 IOS APs radio reset with code 44, mostly seen on 2.4GHz radio CSCvp66546 702w Radio reset due to tx stuck The first one doesn't have a real solution yet, we're running debug images to capture more information. TAC says it might be related to high QBSS, but I saw an AP report high QBSS with no other APs or other known Wi-Fi sources above -80 dBm, so I'm not convinced. The 702W bug has a good debug image in place though. TAC found that stranded packets for clients in power save mode (mostly Murata or Samsung S10 devices) would accumulate in memory and after 60 seconds the AP radio resets to clear the packets. But the reset didn't clear them (or more accumulated) so it just continues resetting the radio. The fixed image discards packets that couldn't be transmitted after 20 seconds, without resetting the radio. It also has a counter that shows the number of times this was triggered and reset prevented Another thing that comes to mind is the issue with MacOS devices sending ARP replies for the gateway IP address. I haven't seen this one, but it sounds similar enough to your issue. https://www.cisco.com/c/en/us/support/docs/wireless/aironet-3800-series-access-points/214491-arp-responses-for-default-gateway-ip-add.html On Mon, Sep 23, 2019, 6:33 PM Jason Cook mailto:jason.c...@adelaide.edu.au>> wrote: We have also experienced AVC crippling our network. Once the traffic volu
RE: [WIRELESS-LAN] Internet Connectivity Issues
Hi Mark, Thanks for the info, I’ll take a look into that. Thanks Sean Sean Gray | B.Sc (Hons) Voice, Collaboration & Wireless Network Analyst ITS, University of Lethbridge From: The EDUCAUSE Wireless Issues Community Group Listserv On Behalf Of Mark Reboli Sent: September 24, 2019 6:58 AM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU Subject: Re: [WIRELESS-LAN] Internet Connectivity Issues Look into the Issue with apple sleeping devices changing their IPs to your gateway. We experienced this causing intermitted issues. Take a look at the link below: https://community.meraki.com/t5/Wireless-LAN/Help-Windows-client-receiving-incorrect-MAC-address-of-Gateway/td-p/39963 If you have any questions please feel free to contact me off line m Mark Reboli Network/Telecom Manager Misericordia University (570) 674-6753 This e-mail and accompanying attachments are confidential. The information is intended solely for the use of the individual to whom it is addressed. Any review, disclosure, copying, distribution, or use of this e-mail communication by others is strictly prohibited. If you are not the intended recipient, please notify us immediately by returning this message to the sender and delete all copies. Thank you for your cooperation. From: The EDUCAUSE Wireless Issues Community Group Listserv [mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU] On Behalf Of Ethan Grinnell Sent: Monday, September 23, 2019 11:04 PM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU<mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU> Subject: Re: [WIRELESS-LAN] Internet Connectivity Issues External Email: Do not click any links or open any attachments unless you trust the sender and know the content is safe. What AP models do you have? We're still on AireOS 8.5.144.33. The COS APs have been running pretty well for us. We've had intermittent issues on IOS APs though. The radios periodically reset and can't seem to clear some packets. Once an AP starts having issues the radio may even cycle every few minutes indefinitely. Do you monitor the AP syslogs? Here are the bug IDs for this: CSCvp86151 IOS APs radio reset with code 44, mostly seen on 2.4GHz radio CSCvp66546 702w Radio reset due to tx stuck The first one doesn't have a real solution yet, we're running debug images to capture more information. TAC says it might be related to high QBSS, but I saw an AP report high QBSS with no other APs or other known Wi-Fi sources above -80 dBm, so I'm not convinced. The 702W bug has a good debug image in place though. TAC found that stranded packets for clients in power save mode (mostly Murata or Samsung S10 devices) would accumulate in memory and after 60 seconds the AP radio resets to clear the packets. But the reset didn't clear them (or more accumulated) so it just continues resetting the radio. The fixed image discards packets that couldn't be transmitted after 20 seconds, without resetting the radio. It also has a counter that shows the number of times this was triggered and reset prevented Another thing that comes to mind is the issue with MacOS devices sending ARP replies for the gateway IP address. I haven't seen this one, but it sounds similar enough to your issue. https://www.cisco.com/c/en/us/support/docs/wireless/aironet-3800-series-access-points/214491-arp-responses-for-default-gateway-ip-add.html On Mon, Sep 23, 2019, 6:33 PM Jason Cook mailto:jason.c...@adelaide.edu.au>> wrote: We have also experienced AVC crippling our network. Once the traffic volumes are higher than X. Performance goes south and can get to 0. Nothing else similar to your description, though that issue was on 8.2 and we are currently 8.5 -- Jason Cook Information Technology and Digital Services The University of Adelaide, AUSTRALIA 5005 --- This email message is intended only for the addressee(s) and contains information which may be confidential and/or copyright. If you are not the intended recipient please do not read, save, forward, disclose, or copy the contents of this email. If this email has been sent to you in error, please notify the sender by reply email and delete this email and any copies or links to this email completely and immediately from your system. No representation is made that this email is free of viruses. Virus scanning is recommended and is the responsibility of the recipient. From: The EDUCAUSE Wireless Issues Community Group Listserv mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU>> On Behalf Of Lee H Badman Sent: Tuesday, 24 September 2019 9:25 AM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU<mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU> Subject: Re: [WIRELESS-LAN] Internet Connectivity Issues Are you running AVC? If so, you might try disabling. It has caused us no end of trouble in the past, similar symptoms. One man’s opinion from past experience. Lee Badman (mobile) On Sep
RE: [WIRELESS-LAN] Internet Connectivity Issues
Hi Ethan, We have a mixture of indoors APs, ranging from a handful of the soon to be replaced 702w, to 1815w, 2702i, 3702i, 2802i & 2802e. No we don’t actively monitor AP syslogs, we are currently trying to utilize DNA Center Assurance for AP / Client health. Our remaining 702w are running very hot, they are sitting at 91% cpu. But that said, we only have a handful, so if these are causing a problem it would be limited to a very small geographic region. Thanks Sean From: The EDUCAUSE Wireless Issues Community Group Listserv On Behalf Of Ethan Grinnell Sent: September 23, 2019 9:04 PM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU Subject: Re: [WIRELESS-LAN] Internet Connectivity Issues What AP models do you have? We're still on AireOS 8.5.144.33. The COS APs have been running pretty well for us. We've had intermittent issues on IOS APs though. The radios periodically reset and can't seem to clear some packets. Once an AP starts having issues the radio may even cycle every few minutes indefinitely. Do you monitor the AP syslogs? Here are the bug IDs for this: CSCvp86151 IOS APs radio reset with code 44, mostly seen on 2.4GHz radio CSCvp66546 702w Radio reset due to tx stuck The first one doesn't have a real solution yet, we're running debug images to capture more information. TAC says it might be related to high QBSS, but I saw an AP report high QBSS with no other APs or other known Wi-Fi sources above -80 dBm, so I'm not convinced. The 702W bug has a good debug image in place though. TAC found that stranded packets for clients in power save mode (mostly Murata or Samsung S10 devices) would accumulate in memory and after 60 seconds the AP radio resets to clear the packets. But the reset didn't clear them (or more accumulated) so it just continues resetting the radio. The fixed image discards packets that couldn't be transmitted after 20 seconds, without resetting the radio. It also has a counter that shows the number of times this was triggered and reset prevented Another thing that comes to mind is the issue with MacOS devices sending ARP replies for the gateway IP address. I haven't seen this one, but it sounds similar enough to your issue. https://www.cisco.com/c/en/us/support/docs/wireless/aironet-3800-series-access-points/214491-arp-responses-for-default-gateway-ip-add.html On Mon, Sep 23, 2019, 6:33 PM Jason Cook mailto:jason.c...@adelaide.edu.au>> wrote: We have also experienced AVC crippling our network. Once the traffic volumes are higher than X. Performance goes south and can get to 0. Nothing else similar to your description, though that issue was on 8.2 and we are currently 8.5 -- Jason Cook Information Technology and Digital Services The University of Adelaide, AUSTRALIA 5005 --- This email message is intended only for the addressee(s) and contains information which may be confidential and/or copyright. If you are not the intended recipient please do not read, save, forward, disclose, or copy the contents of this email. If this email has been sent to you in error, please notify the sender by reply email and delete this email and any copies or links to this email completely and immediately from your system. No representation is made that this email is free of viruses. Virus scanning is recommended and is the responsibility of the recipient. From: The EDUCAUSE Wireless Issues Community Group Listserv mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU>> On Behalf Of Lee H Badman Sent: Tuesday, 24 September 2019 9:25 AM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU<mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU> Subject: Re: [WIRELESS-LAN] Internet Connectivity Issues Are you running AVC? If so, you might try disabling. It has caused us no end of trouble in the past, similar symptoms. One man’s opinion from past experience. Lee Badman (mobile) On Sep 23, 2019, at 7:49 PM, Gray, Sean mailto:sean.gr...@uleth.ca>> wrote: Hi Everyone, We are getting reports of internet connectivity issues from our wireless users. The problem is very temperamental with users bouncing from being able to browse & access App content flawlessly, to experiencing a complete failure to browse to websites and refresh App content. As an example I was able to successfully test Instagram via Safari on an iPhone, and simultaneously fail to see the same content on the Instagram App on the same phone. At this point we are struggling to narrow down the root cause. We have looked at everything from traffic volume to ISP instabilities. But as yet there is no consistent smoking gun. My reason for reaching out to the group is we are running slightly dated code on our HA pair of 5520s. We are running 8.8.111 and plan to upgrade to the latest release as soon as possible. But I’m wondering if anyone out there is running 8.8.111 that have seen or is seeing similar
Re: [WIRELESS-LAN] Internet Connectivity Issues
Hi, We were hit with similar problems across multiple sites on earlier 8.8 builds, now running the latest/ last 8.8.125.3 escalation build from Cisco on multiple controllers with stable results. We are seeing one issue with MBP whereby they remain associated and on session timeout stop forwarding traffic. This is a corner case and yet to be successfully identified as a bug by Cisco TAC. Rgds Carlo On Tue, 24 Sep 2019 at 14:18, John Turner wrote: > I second Mark's suggestion - we've detected this problem at nearly a dozen > of our higher ed and enterprise customers. > > Seems that the issue is between a waking macbook and an android > > > https://www.cisco.com/c/en/us/support/docs/wireless/aironet-3800-series-access-points/214491-arp-responses-for-default-gateway-ip-add.html > > Traditional tools aren't finding this - we are detecting a drop in traffic > as well as the duplicate gateway address that triggers an advisory alert. > > John > > On Tue, Sep 24, 2019 at 8:57 AM Mark Reboli > wrote: > >> Look into the Issue with apple sleeping devices changing their IPs to >> your gateway. We experienced this causing intermitted issues. Take a look >> at the link below: >> >> >> >> >> https://community.meraki.com/t5/Wireless-LAN/Help-Windows-client-receiving-incorrect-MAC-address-of-Gateway/td-p/39963 >> >> >> >> If you have any questions please feel free to contact me off line >> >> >> >> m >> >> Mark Reboli >> >> Network/Telecom Manager >> >> Misericordia University >> >> (570) 674-6753 >> >> >> >> This e-mail and accompanying attachments are confidential. The >> information is intended solely for the use of the individual to whom it is >> addressed. Any review, disclosure, copying, distribution, or use of this >> e-mail communication by others is strictly prohibited. If you are not the >> intended recipient, please notify us immediately by returning this message >> to the sender and delete all copies. Thank you for your cooperation. >> >> >> >> >> >> >> >> >> >> *From:* The EDUCAUSE Wireless Issues Community Group Listserv [mailto: >> WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU] *On Behalf Of *Ethan Grinnell >> *Sent:* Monday, September 23, 2019 11:04 PM >> *To:* WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU >> *Subject:* Re: [WIRELESS-LAN] Internet Connectivity Issues >> >> >> >> *External Email:* Do not click any links or open any attachments unless >> you trust the sender and know the content is safe. >> >> What AP models do you have? >> >> >> >> We're still on AireOS 8.5.144.33. The COS APs have been running pretty >> well for us. >> >> >> >> We've had intermittent issues on IOS APs though. The radios periodically >> reset and can't seem to clear some packets. Once an AP starts having issues >> the radio may even cycle every few minutes indefinitely. Do you monitor the >> AP syslogs? >> >> >> >> Here are the bug IDs for this: >> >> *CSCvp86151 IOS APs radio reset with code 44, mostly seen on 2.4GHz radio* >> >> *CSCvp66546 702w Radio reset due to tx stuck* >> >> >> >> The first one doesn't have a real solution yet, we're running debug >> images to capture more information. TAC says it might be related to high >> QBSS, but I saw an AP report high QBSS with no other APs or other known >> Wi-Fi sources above -80 dBm, so I'm not convinced. >> >> >> >> The 702W bug has a good debug image in place though. TAC found that >> stranded packets for clients in power save mode (mostly Murata or Samsung >> S10 devices) would accumulate in memory and after 60 seconds the AP radio >> resets to clear the packets. But the reset didn't clear them (or more >> accumulated) so it just continues resetting the radio. The fixed image >> discards packets that couldn't be transmitted after 20 seconds, without >> resetting the radio. It also has a counter that shows the number of times >> this was triggered and reset prevented >> >> >> >> Another thing that comes to mind is the issue with MacOS devices sending >> ARP replies for the gateway IP address. I haven't seen this one, but it >> sounds similar enough to your issue. >> >> >> >> >> https://www.cisco.com/c/en/us/support/docs/wireless/aironet-3800-series-access-points/214491-arp-responses-for-default-gateway-ip-add.html >> >> >> >> >> >> On M
RE: [WIRELESS-LAN] Internet Connectivity Issues
Hi Lee, Thanks for the suggestion. We had Application Visibility enabled at the WLAN level, but we weren’t applying any kind of control in the form of an AVC profile. I’ve turned it off to see if that improves things. Thanks Sean Sean Gray | B.Sc (Hons) Voice, Collaboration & Wireless Network Analyst ITS, University of Lethbridge From: The EDUCAUSE Wireless Issues Community Group Listserv On Behalf Of Lee H Badman Sent: September 23, 2019 5:55 PM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU Subject: Re: [WIRELESS-LAN] Internet Connectivity Issues Are you running AVC? If so, you might try disabling. It has caused us no end of trouble in the past, similar symptoms. One man’s opinion from past experience. Lee Badman (mobile) On Sep 23, 2019, at 7:49 PM, Gray, Sean mailto:sean.gr...@uleth.ca>> wrote: Hi Everyone, We are getting reports of internet connectivity issues from our wireless users. The problem is very temperamental with users bouncing from being able to browse & access App content flawlessly, to experiencing a complete failure to browse to websites and refresh App content. As an example I was able to successfully test Instagram via Safari on an iPhone, and simultaneously fail to see the same content on the Instagram App on the same phone. At this point we are struggling to narrow down the root cause. We have looked at everything from traffic volume to ISP instabilities. But as yet there is no consistent smoking gun. My reason for reaching out to the group is we are running slightly dated code on our HA pair of 5520s. We are running 8.8.111 and plan to upgrade to the latest release as soon as possible. But I’m wondering if anyone out there is running 8.8.111 that have seen or is seeing similar issues. Thanks Sean ** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://www.educause.edu/community ** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://www.educause.edu/community ** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://www.educause.edu/community
Re: [WIRELESS-LAN] Internet Connectivity Issues
I second Mark's suggestion - we've detected this problem at nearly a dozen of our higher ed and enterprise customers. Seems that the issue is between a waking macbook and an android https://www.cisco.com/c/en/us/support/docs/wireless/aironet-3800-series-access-points/214491-arp-responses-for-default-gateway-ip-add.html Traditional tools aren't finding this - we are detecting a drop in traffic as well as the duplicate gateway address that triggers an advisory alert. John On Tue, Sep 24, 2019 at 8:57 AM Mark Reboli wrote: > Look into the Issue with apple sleeping devices changing their IPs to your > gateway. We experienced this causing intermitted issues. Take a look at > the link below: > > > > > https://community.meraki.com/t5/Wireless-LAN/Help-Windows-client-receiving-incorrect-MAC-address-of-Gateway/td-p/39963 > > > > If you have any questions please feel free to contact me off line > > > > m > > Mark Reboli > > Network/Telecom Manager > > Misericordia University > > (570) 674-6753 > > > > This e-mail and accompanying attachments are confidential. The > information is intended solely for the use of the individual to whom it is > addressed. Any review, disclosure, copying, distribution, or use of this > e-mail communication by others is strictly prohibited. If you are not the > intended recipient, please notify us immediately by returning this message > to the sender and delete all copies. Thank you for your cooperation. > > > > > > > > > > *From:* The EDUCAUSE Wireless Issues Community Group Listserv [mailto: > WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU] *On Behalf Of *Ethan Grinnell > *Sent:* Monday, September 23, 2019 11:04 PM > *To:* WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU > *Subject:* Re: [WIRELESS-LAN] Internet Connectivity Issues > > > > *External Email:* Do not click any links or open any attachments unless > you trust the sender and know the content is safe. > > What AP models do you have? > > > > We're still on AireOS 8.5.144.33. The COS APs have been running pretty > well for us. > > > > We've had intermittent issues on IOS APs though. The radios periodically > reset and can't seem to clear some packets. Once an AP starts having issues > the radio may even cycle every few minutes indefinitely. Do you monitor the > AP syslogs? > > > > Here are the bug IDs for this: > > *CSCvp86151 IOS APs radio reset with code 44, mostly seen on 2.4GHz radio* > > *CSCvp66546 702w Radio reset due to tx stuck* > > > > The first one doesn't have a real solution yet, we're running debug images > to capture more information. TAC says it might be related to high QBSS, but > I saw an AP report high QBSS with no other APs or other known Wi-Fi sources > above -80 dBm, so I'm not convinced. > > > > The 702W bug has a good debug image in place though. TAC found that > stranded packets for clients in power save mode (mostly Murata or Samsung > S10 devices) would accumulate in memory and after 60 seconds the AP radio > resets to clear the packets. But the reset didn't clear them (or more > accumulated) so it just continues resetting the radio. The fixed image > discards packets that couldn't be transmitted after 20 seconds, without > resetting the radio. It also has a counter that shows the number of times > this was triggered and reset prevented > > > > Another thing that comes to mind is the issue with MacOS devices sending > ARP replies for the gateway IP address. I haven't seen this one, but it > sounds similar enough to your issue. > > > > > https://www.cisco.com/c/en/us/support/docs/wireless/aironet-3800-series-access-points/214491-arp-responses-for-default-gateway-ip-add.html > > > > > > On Mon, Sep 23, 2019, 6:33 PM Jason Cook > wrote: > > We have also experienced AVC crippling our network. Once the traffic > volumes are higher than X. Performance goes south and can get to 0. > > > > Nothing else similar to your description, though that issue was on 8.2 and > we are currently 8.5 > > > > > > > > -- > > Jason Cook > > Information Technology and Digital Services > > The University of Adelaide, AUSTRALIA 5005 > > --- > > This email message is intended only for the addressee(s) and contains > information which may be confidential and/or copyright. If you are not the > intended recipient please do not read, save, forward, disclose, or copy the > contents of this email. If this email has been sent to you in error, please > notify the sender by reply email and delete this email and any copie
RE: [WIRELESS-LAN] Internet Connectivity Issues
Look into the Issue with apple sleeping devices changing their IPs to your gateway. We experienced this causing intermitted issues. Take a look at the link below: https://community.meraki.com/t5/Wireless-LAN/Help-Windows-client-receiving-incorrect-MAC-address-of-Gateway/td-p/39963 If you have any questions please feel free to contact me off line m Mark Reboli Network/Telecom Manager Misericordia University (570) 674-6753 This e-mail and accompanying attachments are confidential. The information is intended solely for the use of the individual to whom it is addressed. Any review, disclosure, copying, distribution, or use of this e-mail communication by others is strictly prohibited. If you are not the intended recipient, please notify us immediately by returning this message to the sender and delete all copies. Thank you for your cooperation. From: The EDUCAUSE Wireless Issues Community Group Listserv [mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU] On Behalf Of Ethan Grinnell Sent: Monday, September 23, 2019 11:04 PM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU Subject: Re: [WIRELESS-LAN] Internet Connectivity Issues External Email: Do not click any links or open any attachments unless you trust the sender and know the content is safe. What AP models do you have? We're still on AireOS 8.5.144.33. The COS APs have been running pretty well for us. We've had intermittent issues on IOS APs though. The radios periodically reset and can't seem to clear some packets. Once an AP starts having issues the radio may even cycle every few minutes indefinitely. Do you monitor the AP syslogs? Here are the bug IDs for this: CSCvp86151 IOS APs radio reset with code 44, mostly seen on 2.4GHz radio CSCvp66546 702w Radio reset due to tx stuck The first one doesn't have a real solution yet, we're running debug images to capture more information. TAC says it might be related to high QBSS, but I saw an AP report high QBSS with no other APs or other known Wi-Fi sources above -80 dBm, so I'm not convinced. The 702W bug has a good debug image in place though. TAC found that stranded packets for clients in power save mode (mostly Murata or Samsung S10 devices) would accumulate in memory and after 60 seconds the AP radio resets to clear the packets. But the reset didn't clear them (or more accumulated) so it just continues resetting the radio. The fixed image discards packets that couldn't be transmitted after 20 seconds, without resetting the radio. It also has a counter that shows the number of times this was triggered and reset prevented Another thing that comes to mind is the issue with MacOS devices sending ARP replies for the gateway IP address. I haven't seen this one, but it sounds similar enough to your issue. https://www.cisco.com/c/en/us/support/docs/wireless/aironet-3800-series-access-points/214491-arp-responses-for-default-gateway-ip-add.html On Mon, Sep 23, 2019, 6:33 PM Jason Cook mailto:jason.c...@adelaide.edu.au>> wrote: We have also experienced AVC crippling our network. Once the traffic volumes are higher than X. Performance goes south and can get to 0. Nothing else similar to your description, though that issue was on 8.2 and we are currently 8.5 -- Jason Cook Information Technology and Digital Services The University of Adelaide, AUSTRALIA 5005 --- This email message is intended only for the addressee(s) and contains information which may be confidential and/or copyright. If you are not the intended recipient please do not read, save, forward, disclose, or copy the contents of this email. If this email has been sent to you in error, please notify the sender by reply email and delete this email and any copies or links to this email completely and immediately from your system. No representation is made that this email is free of viruses. Virus scanning is recommended and is the responsibility of the recipient. From: The EDUCAUSE Wireless Issues Community Group Listserv mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU>> On Behalf Of Lee H Badman Sent: Tuesday, 24 September 2019 9:25 AM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU<mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU> Subject: Re: [WIRELESS-LAN] Internet Connectivity Issues Are you running AVC? If so, you might try disabling. It has caused us no end of trouble in the past, similar symptoms. One man’s opinion from past experience. Lee Badman (mobile) On Sep 23, 2019, at 7:49 PM, Gray, Sean mailto:sean.gr...@uleth.ca>> wrote: Hi Everyone, We are getting reports of internet connectivity issues from our wireless users. The problem is very temperamental with users bouncing from being able to browse & access App content flawlessly, to experiencing a complete failure to browse to websites and refresh App content. As an example I was able to successfully test Instagram via Safari on an iPhone,
Re: [WIRELESS-LAN] Internet Connectivity Issues
What AP models do you have? We're still on AireOS 8.5.144.33. The COS APs have been running pretty well for us. We've had intermittent issues on IOS APs though. The radios periodically reset and can't seem to clear some packets. Once an AP starts having issues the radio may even cycle every few minutes indefinitely. Do you monitor the AP syslogs? Here are the bug IDs for this: *CSCvp86151 IOS APs radio reset with code 44, mostly seen on 2.4GHz radio* *CSCvp66546 702w Radio reset due to tx stuck* The first one doesn't have a real solution yet, we're running debug images to capture more information. TAC says it might be related to high QBSS, but I saw an AP report high QBSS with no other APs or other known Wi-Fi sources above -80 dBm, so I'm not convinced. The 702W bug has a good debug image in place though. TAC found that stranded packets for clients in power save mode (mostly Murata or Samsung S10 devices) would accumulate in memory and after 60 seconds the AP radio resets to clear the packets. But the reset didn't clear them (or more accumulated) so it just continues resetting the radio. The fixed image discards packets that couldn't be transmitted after 20 seconds, without resetting the radio. It also has a counter that shows the number of times this was triggered and reset prevented Another thing that comes to mind is the issue with MacOS devices sending ARP replies for the gateway IP address. I haven't seen this one, but it sounds similar enough to your issue. https://www.cisco.com/c/en/us/support/docs/wireless/aironet-3800-series-access-points/214491-arp-responses-for-default-gateway-ip-add.html On Mon, Sep 23, 2019, 6:33 PM Jason Cook wrote: > We have also experienced AVC crippling our network. Once the traffic > volumes are higher than X. Performance goes south and can get to 0. > > > > Nothing else similar to your description, though that issue was on 8.2 and > we are currently 8.5 > > > > > > > > -- > > Jason Cook > > Information Technology and Digital Services > > The University of Adelaide, AUSTRALIA 5005 > > --- > > This email message is intended only for the addressee(s) and contains > information which may be confidential and/or copyright. If you are not the > intended recipient please do not read, save, forward, disclose, or copy the > contents of this email. If this email has been sent to you in error, please > notify the sender by reply email and delete this email and any copies or > links to this email completely and immediately from your system. No > representation is made that this email is free of viruses. Virus scanning > is recommended and is the responsibility of the recipient. > > > > *From:* The EDUCAUSE Wireless Issues Community Group Listserv < > WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU> *On Behalf Of *Lee H Badman > *Sent:* Tuesday, 24 September 2019 9:25 AM > *To:* WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU > *Subject:* Re: [WIRELESS-LAN] Internet Connectivity Issues > > > > Are you running AVC? If so, you might try disabling. It has caused us no > end of trouble in the past, similar symptoms. One man’s opinion from past > experience. > > Lee Badman (mobile) > > > On Sep 23, 2019, at 7:49 PM, Gray, Sean wrote: > > Hi Everyone, > > > > We are getting reports of internet connectivity issues from our wireless > users. The problem is very temperamental with users bouncing from being > able to browse & access App content flawlessly, to experiencing a complete > failure to browse to websites and refresh App content. As an example I was > able to successfully test Instagram via Safari on an iPhone, and > simultaneously fail to see the same content on the Instagram App on the > same phone. > > > > At this point we are struggling to narrow down the root cause. We have > looked at everything from traffic volume to ISP instabilities. But as yet > there is no consistent smoking gun. My reason for reaching out to the group > is we are running slightly dated code on our HA pair of 5520s. We are > running 8.8.111 and plan to upgrade to the latest release as soon as > possible. But I’m wondering if anyone out there is running 8.8.111 that > have seen or is seeing similar issues. > > > > Thanks > > > > Sean > > ** > Replies to EDUCAUSE Community Group emails are sent to the entire > community list. If you want to reply only to the person who sent the > message, copy and paste their email address and forward the email reply. > Additional participation and subscription information can be found at > https://www.educause.edu/community > > ** > Replies to EDUCAUSE Community Group emails are sent to t
RE: [WIRELESS-LAN] Internet Connectivity Issues
We have also experienced AVC crippling our network. Once the traffic volumes are higher than X. Performance goes south and can get to 0. Nothing else similar to your description, though that issue was on 8.2 and we are currently 8.5 -- Jason Cook Information Technology and Digital Services The University of Adelaide, AUSTRALIA 5005 --- This email message is intended only for the addressee(s) and contains information which may be confidential and/or copyright. If you are not the intended recipient please do not read, save, forward, disclose, or copy the contents of this email. If this email has been sent to you in error, please notify the sender by reply email and delete this email and any copies or links to this email completely and immediately from your system. No representation is made that this email is free of viruses. Virus scanning is recommended and is the responsibility of the recipient. From: The EDUCAUSE Wireless Issues Community Group Listserv On Behalf Of Lee H Badman Sent: Tuesday, 24 September 2019 9:25 AM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU Subject: Re: [WIRELESS-LAN] Internet Connectivity Issues Are you running AVC? If so, you might try disabling. It has caused us no end of trouble in the past, similar symptoms. One man’s opinion from past experience. Lee Badman (mobile) On Sep 23, 2019, at 7:49 PM, Gray, Sean mailto:sean.gr...@uleth.ca>> wrote: Hi Everyone, We are getting reports of internet connectivity issues from our wireless users. The problem is very temperamental with users bouncing from being able to browse & access App content flawlessly, to experiencing a complete failure to browse to websites and refresh App content. As an example I was able to successfully test Instagram via Safari on an iPhone, and simultaneously fail to see the same content on the Instagram App on the same phone. At this point we are struggling to narrow down the root cause. We have looked at everything from traffic volume to ISP instabilities. But as yet there is no consistent smoking gun. My reason for reaching out to the group is we are running slightly dated code on our HA pair of 5520s. We are running 8.8.111 and plan to upgrade to the latest release as soon as possible. But I’m wondering if anyone out there is running 8.8.111 that have seen or is seeing similar issues. Thanks Sean ** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://www.educause.edu/community ** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://www.educause.edu/community ** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://www.educause.edu/community
Re: [WIRELESS-LAN] Internet Connectivity Issues
Are you running AVC? If so, you might try disabling. It has caused us no end of trouble in the past, similar symptoms. One man’s opinion from past experience. Lee Badman (mobile) On Sep 23, 2019, at 7:49 PM, Gray, Sean mailto:sean.gr...@uleth.ca>> wrote: Hi Everyone, We are getting reports of internet connectivity issues from our wireless users. The problem is very temperamental with users bouncing from being able to browse & access App content flawlessly, to experiencing a complete failure to browse to websites and refresh App content. As an example I was able to successfully test Instagram via Safari on an iPhone, and simultaneously fail to see the same content on the Instagram App on the same phone. At this point we are struggling to narrow down the root cause. We have looked at everything from traffic volume to ISP instabilities. But as yet there is no consistent smoking gun. My reason for reaching out to the group is we are running slightly dated code on our HA pair of 5520s. We are running 8.8.111 and plan to upgrade to the latest release as soon as possible. But I’m wondering if anyone out there is running 8.8.111 that have seen or is seeing similar issues. Thanks Sean ** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://www.educause.edu/community ** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://www.educause.edu/community
Internet Connectivity Issues
Hi Everyone, We are getting reports of internet connectivity issues from our wireless users. The problem is very temperamental with users bouncing from being able to browse & access App content flawlessly, to experiencing a complete failure to browse to websites and refresh App content. As an example I was able to successfully test Instagram via Safari on an iPhone, and simultaneously fail to see the same content on the Instagram App on the same phone. At this point we are struggling to narrow down the root cause. We have looked at everything from traffic volume to ISP instabilities. But as yet there is no consistent smoking gun. My reason for reaching out to the group is we are running slightly dated code on our HA pair of 5520s. We are running 8.8.111 and plan to upgrade to the latest release as soon as possible. But I'm wondering if anyone out there is running 8.8.111 that have seen or is seeing similar issues. Thanks Sean ** Replies to EDUCAUSE Community Group emails are sent to the entire community list. If you want to reply only to the person who sent the message, copy and paste their email address and forward the email reply. Additional participation and subscription information can be found at https://www.educause.edu/community
