RE: Move In/Opening Week- Any Problems?

[Hector J Rios]

Thank you for the good thoughts on the storm. Luckily we are fine.

So far we’ve been told that the issue we experienced was a combination of two 
things: 1) the 8540’s memory queues and buffers reached their maximum capacity. 
This affected both 802.1X and CAPWAP. Thus the AP flapping. 2) RADIUS and EAP 
timers must be EXTRA optimized. I say EXTRA, because we’ve always followed best 
practices and recommendations from TAC.

This is a good document to read: 
https://www.cisco.com/c/en/us/support/docs/wireless-mobility/wireless-lan-wlan/118703-technote-wlc-00.html

Finally, what is most interesting is the fact that even though the 8540 is 
advertised to support 6000 APs and 64000 clients, these numbers do not seem to 
be valid if your environment is mainly 802.1X. So, if your environment is 
mainly 802.1X, and you have an 8540, I would recommend you talk to your Cisco 
SE so they can tell you what the official supported number of APs is. I’ve yet 
to find any official documentation that even hints to this. Miercom performed a 
comparative test in 2015 between Aruba and Cisco, and in the report they did 
test client authentication rate, but only for the Cisco 5520.

https://www.cisco.com/c/dam/en/us/products/collateral/wireless/8540-wireless-controller/miercom-report-wlcs-cisco-aruba.pdf

TAC’s recommendation is for us to use 8.2.160 on the 8540s. We will make all 
necessary config changes and start moving APs in waves of 500 slowly so we can 
watch utilization. Our plan also includes not to exceed the AP capacity of the 
8540s by 50%-60%. If this works, we will have to get an additional pair of 
8540s. I’ll let you know if we are successful.

BTW, we require to have AVC turned on. TAC is very concerned about this. We’ll 
also be watching this.

-Hector

From: The EDUCAUSE Wireless Issues Constituent Group Listserv 
[mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU] On Behalf Of Lee H Badman
Sent: Wednesday, August 30, 2017 6:43 AM
To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU
Subject: Re: [WIRELESS-LAN] Move In/Opening Week- Any Problems?


Hi Hector,



I hope the storm is not causing havoc for you down there- good thoughts to you 
on that.



Did you get anywhere with Cisco on your 8540/8.2.160 problems? I'm being told 
we may need to go that same combination and it doesn't inspire confidence.



Evidently my 8.2.151 (you know... one of those STABLE code versions) may be a 
time bomb that caused a spontaneous 8540 reboot. The comment was made that our 
3300 APs on a platform that supposedly supports 6000 somehow equals a dense 
deployment and that we likely are hitting:

___
Regarding the logs, I was able to check the logs, and yes It seems your 
deployment is a high-density deployment with over 3000 APs.

Based on your deployment and the logs I was able to identify this

It seems the WLC is having load process utilization  on the task SpamReceive 
Task and HAConfigSyncTask.

spamApTask15992   ( 53/ 78)0 (  0/  0)%  30   22
 spamApTask05991   ( 72/ 70)0 (  0/  0)%  305
 spamReceiveTask5990   ( 52/ 78)0 (  0/  0)%  990
 spamSocketTask 5989   (175/ 32)0 (  0/  0)%   0   13
 HAPeerToPeerCommTa 5988   ( 90/ 64)0 (  0/  0)%   07
 rmgrPing   5987   ( 80/ 67)0 (  0/  0)%   0   13

HAConfigSyncTask   6204   (240/  7)0 (  0/  0)%  993
​
Based on the symptoms, the WLC version and your WLC density. You may be hitting 
bug.

CSCvd20251 - Data Plane stopped working on Cisco 5508 WLC running 
8.0.140.0
 ___
I hope to have confirmation today. I can't imagine what Cisco could have done 
between .151 and .6 to make this sort of thing better, and I am really 
interested in whether they isolated your own .160 problems. There is no way in 
hell I'm moving to that version without seeing case notes on every single issue 
people are having in this continual cycle of trading one set of bugs for 
another.

This game just isn't fun anymore.

Thanks-




Lee Badman | Network Architect | CWNE #200
Information Technology Services
206 Machinery Hall
120 Smith Drive
Syracuse, New York 13244
t 315.443.3003   f 315.443.4325   e lhbad...@syr.edu w 
its.syr.edu
SYRACUSE UNIVERSITY
syr.edu

From: The EDUCAUSE Wireless Issues Constituent Group Listserv 
mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU>> 
on behalf of Hector J Rios mailto:hr...@lsu.edu>>
Sent: Friday, August 25, 2017 3:11 PM
To: 
WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU
Subject: Re: [WIRELESS-LAN] Move In/Opening Week- Any Problems?

Here’s ours:

2 8540s in HA mode (bought with the idea of replacing all WiSM2s)
4 pairs of WiSM2s in HA mode
3 server ClearPass cluster for both eduroam and guest
Main SSID: eduroam with PEAP/MSCHAP
Mix of WAPs; 3500, 3600, 3700, 2800, 1810w
Total number of WAPs: 35

Re: Move In/Opening Week- Any Problems?

[Lee H Badman]

Hi Hector,


I hope the storm is not causing havoc for you down there- good thoughts to you 
on that.


Did you get anywhere with Cisco on your 8540/8.2.160 problems? I'm being told 
we may need to go that same combination and it doesn't inspire confidence.


Evidently my 8.2.151 (you know... one of those STABLE code versions) may be a 
time bomb that caused a spontaneous 8540 reboot. The comment was made that our 
3300 APs on a platform that supposedly supports 6000 somehow equals a dense 
deployment and that we likely are hitting:

___

Regarding the logs, I was able to check the logs, and yes It seems your 
deployment is a high-density deployment with over 3000 APs.

Based on your deployment and the logs I was able to identify this

It seems the WLC is having load process utilization  on the task SpamReceive 
Task and HAConfigSyncTask.

spamApTask15992   ( 53/ 78)0 (  0/  0)%  30   22
 spamApTask05991   ( 72/ 70)0 (  0/  0)%  305
 spamReceiveTask5990   ( 52/ 78)0 (  0/  0)%  990
 spamSocketTask 5989   (175/ 32)0 (  0/  0)%   0   13
 HAPeerToPeerCommTa 5988   ( 90/ 64)0 (  0/  0)%   07
 rmgrPing   5987   ( 80/ 67)0 (  0/  0)%   0   13

HAConfigSyncTask   6204   (240/  7)0 (  0/  0)%  993
​
Based on the symptoms, the WLC version and your WLC density. You may be hitting 
bug.

CSCvd20251 - Data Plane stopped working on Cisco 5508 WLC running 
8.0.140.0
 ___
I hope to have confirmation today. I can't imagine what Cisco could have done 
between .151 and .6 to make this sort of thing better, and I am really 
interested in whether they isolated your own .160 problems. There is no way in 
hell I'm moving to that version without seeing case notes on every single issue 
people are having in this continual cycle of trading one set of bugs for 
another.

This game just isn't fun anymore.

Thanks-



Lee Badman | Network Architect | CWNE #200
Information Technology Services
206 Machinery Hall
120 Smith Drive
Syracuse, New York 13244
t 315.443.3003   f 315.443.4325   e lhbad...@syr.edu w 
its.syr.edu
SYRACUSE UNIVERSITY
syr.edu

From: The EDUCAUSE Wireless Issues Constituent Group Listserv 
 on behalf of Hector J Rios 
Sent: Friday, August 25, 2017 3:11 PM
To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU
Subject: Re: [WIRELESS-LAN] Move In/Opening Week- Any Problems?

Here’s ours:

2 8540s in HA mode (bought with the idea of replacing all WiSM2s)
4 pairs of WiSM2s in HA mode
3 server ClearPass cluster for both eduroam and guest
Main SSID: eduroam with PEAP/MSCHAP
Mix of WAPs; 3500, 3600, 3700, 2800, 1810w
Total number of WAPs: 3500
21000 peak users

We tested the 8540s extensively over the spring and summer, primarily with the 
8.2.151 code and a mix of 2800s and 1810ws. We had AVC turned on, and were 
using RLANs for the wired ports. The largest number of WAPs we had on this pair 
was 469. We tested code 8.2.160 towards the end of the summer with all WAPs on 
the 8540s, and had no issues. First the day of classes, we had all WiSM2s 
running 8.2.160 simply as a backup. Early morning we started getting reports of 
802.1X authentication failures (these failures had nothing to do with 
ClearPass). Shortly after that, WAPs starting flapping (disconnecting from the 
8540s moving to WiSM2s and then moving back again). We tried playing with the 
TCP MSS setting, adjusting EAP timers, turning AVC off and multiple other 
things, but nothing worked. In the end, we downgraded the WiSM2s to 8.0.140 and 
moved all WAPs that were not 2800 or 1810s. The 8540s were downgraded to 
8.2.151 so the 2800s and 1810s would have a controller to connect to. Network 
stability was restored after this.

Needless to say it was a very unpleasant experience. We are still working with 
Cisco to find out the root cause of the problem.

Hector Rios
Louisiana State University

From: The EDUCAUSE Wireless Issues Constituent Group Listserv 
[mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU] On Behalf Of Lee H Badman
Sent: Friday, August 25, 2017 8:22 AM
To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU
Subject: [WIRELESS-LAN] Move In/Opening Week- Any Problems?

It might be beneficial to share notes in case other schools are hitting common 
problems. I’m wondering how everyone who is in the thick of it is faring with 
back-to-school?

On this end, we are doing OK halfway to our expected total daily peak clients 
(we’re at 15K now high water mark).

Our significant WLAN-related changes since end of Spring semester
· Running 8.2.151 on our 8540s
· Significant quantities of Wave 2 APs
· ISE as RADIUS (only, no NAC, no onboarding)

No changes to:
· our guest WLAN (Clearpass/an Aruba controller pair)
· onboarding (Cloudpath Wiz)
· overall topology
· open network in

RE: Move In/Opening Week- Any Problems?

[Osborne, Bruce W (Network Operations)]

Good call on the multiple helpers.

Several years ago we were using Cisco HSRP on some subnets with 2 SVIs. I had a 
testing issue with an external wired NIC in a game console. With HSRP, both 
SVIs use their helpers to respond to DHCP requests. The client basically gets 2 
identical offers from the same server. This client would accept the first offer 
& reject the second. This meant the client would not get an ip address. For the 
duration of my testing, removed HSRP from this subnet, using a single SVI.


Bruce Osborne
Senior Network Engineer
Network Operations - Wireless
 (434) 592-4229
LIBERTY UNIVERSITY
Training Champions for Christ since 1971

From: Michael Dickson [mailto:mdick...@nic.umass.edu]
Sent: Saturday, August 26, 2017 4:05 PM
Subject: Re: Move In/Opening Week- Any Problems?

Just a thought but do you have multiple helper addresses configured for that 
vlan/subnet? I'm wondering if maybe the printers aren't expecting that. Another 
random thought, if they're not broadcasting for a lease because they require a 
static could they have maybe all self-assigned themselves the same IP and are 
discovering each other over L2?

Good luck. We're pretty much going down the same CPPM/Airgroup path right now.

Mike

Michael Dickson
Network Engineer
Information Technology
University of Massachusetts Amherst
413-545-9639
michael.dick...@umass.edu<mailto:michael.dick...@umass.edu>
PGP: 0x16777D39

On Aug 26, 2017, at 3:18 PM, Michael Davis 
mailto:da...@udel.edu>> wrote:
First Semester supporting mDNS in production with Aruba Clearpass Airgroup.

Almost every Epson XP series printer is complaining of duplicate IP addresses
which of course is not the case.  Anyone see anything similar?  There are a few
older web searches about Epson's requiring a static IP, which isn't an option 
right
now unfortunately.

Only Freshmen moving in today (~5K), the bulk (~20K) will arrive tomorrow and
throughout the week.

ArubaOS 6.5.3.2
CPPM 6.6.7.96909
Four 7240 controllers
~3200 APs
Three primary SSIDs: eduroam, Devices, Guest (clearpass)


thanks
mike

On 8/25/17 9:22 AM, Lee H Badman wrote:
It might be beneficial to share notes in case other schools are hitting common 
problems. I’m wondering how everyone who is in the thick of it is faring with 
back-to-school?

On this end, we are doing OK halfway to our expected total daily peak clients 
(we’re at 15K now high water mark).

Our significant WLAN-related changes since end of Spring semester
·Running 8.2.151 on our 8540s
·Significant quantities of Wave 2 APs
·ISE as RADIUS (only, no NAC, no onboarding)

No changes to:
·our guest WLAN (Clearpass/an Aruba controller pair)
·onboarding (Cloudpath Wiz)
·overall topology
·open network in dorms for gadgets
·non-use of AVC, it crapped out and never got solved after hundreds of 
hours with TAC

Fears:
·We haven’t yet hit the scale that will reveal problems with any of the 
newer stuff listed above

Anyone else care to share?

-Lee


Lee Badman | Network Architect

Certified Wireless Network Expert (#200)
Information Technology Services
206 Machinery Hall
120 Smith Drive
Syracuse, New York 13244
t 315.443.3003   f 315.443.4325   e lhbad...@syr.edu<mailto:lhbad...@syr.edu> w 
its.syr.edu<http://its.syr.edu>
SYRACUSE UNIVERSITY
syr.edu<http://syr.edu>



** Participation and subscription information for this EDUCAUSE 
Constituent Group discussion list can be found at 
http://www.educause.edu/discuss.


** Participation and subscription information for this EDUCAUSE 
Constituent Group discussion list can be found at 
http://www.educause.edu/discuss.
** Participation and subscription information for this EDUCAUSE 
Constituent Group discussion list can be found at 
http://www.educause.edu/discuss.

**
Participation and subscription information for this EDUCAUSE Constituent Group 
discussion list can be found at http://www.educause.edu/discuss.

RE: Move In/Opening Week- Any Problems?

[Entwistle, Bruce]

We have also recently upgraded to 8.2.160.0 on our pair of 5508 controllers, 
and students are scheduled to begin arriving next week.  Any comments from 
Cisco regarding the 8.2.160.0 code?

Bruce Entwistle
Network Manager
University of Redlands


From: The EDUCAUSE Wireless Issues Constituent Group Listserv 
[mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU] On Behalf Of Hector J Rios
Sent: Friday, August 25, 2017 12:12 PM
To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU
Subject: Re: [WIRELESS-LAN] Move In/Opening Week- Any Problems?

Here's ours:

2 8540s in HA mode (bought with the idea of replacing all WiSM2s)
4 pairs of WiSM2s in HA mode
3 server ClearPass cluster for both eduroam and guest
Main SSID: eduroam with PEAP/MSCHAP
Mix of WAPs; 3500, 3600, 3700, 2800, 1810w
Total number of WAPs: 3500
21000 peak users

We tested the 8540s extensively over the spring and summer, primarily with the 
8.2.151 code and a mix of 2800s and 1810ws. We had AVC turned on, and were 
using RLANs for the wired ports. The largest number of WAPs we had on this pair 
was 469. We tested code 8.2.160 towards the end of the summer with all WAPs on 
the 8540s, and had no issues. First the day of classes, we had all WiSM2s 
running 8.2.160 simply as a backup. Early morning we started getting reports of 
802.1X authentication failures (these failures had nothing to do with 
ClearPass). Shortly after that, WAPs starting flapping (disconnecting from the 
8540s moving to WiSM2s and then moving back again). We tried playing with the 
TCP MSS setting, adjusting EAP timers, turning AVC off and multiple other 
things, but nothing worked. In the end, we downgraded the WiSM2s to 8.0.140 and 
moved all WAPs that were not 2800 or 1810s. The 8540s were downgraded to 
8.2.151 so the 2800s and 1810s would have a controller to connect to. Network 
stability was restored after this.

Needless to say it was a very unpleasant experience. We are still working with 
Cisco to find out the root cause of the problem.

Hector Rios
Louisiana State University

From: The EDUCAUSE Wireless Issues Constituent Group Listserv 
[mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU] On Behalf Of Lee H Badman
Sent: Friday, August 25, 2017 8:22 AM
To: 
WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU
Subject: [WIRELESS-LAN] Move In/Opening Week- Any Problems?

It might be beneficial to share notes in case other schools are hitting common 
problems. I'm wondering how everyone who is in the thick of it is faring with 
back-to-school?

On this end, we are doing OK halfway to our expected total daily peak clients 
(we're at 15K now high water mark).

Our significant WLAN-related changes since end of Spring semester
* Running 8.2.151 on our 8540s
* Significant quantities of Wave 2 APs
* ISE as RADIUS (only, no NAC, no onboarding)

No changes to:
* our guest WLAN (Clearpass/an Aruba controller pair)
* onboarding (Cloudpath Wiz)
* overall topology
* open network in dorms for gadgets
* non-use of AVC, it crapped out and never got solved after hundreds of 
hours with TAC

Fears:
* We haven't yet hit the scale that will reveal problems with any of 
the newer stuff listed above

Anyone else care to share?

-Lee


Lee Badman | Network Architect

Certified Wireless Network Expert (#200)
Information Technology Services
206 Machinery Hall
120 Smith Drive
Syracuse, New York 13244
t 315.443.3003   f 315.443.4325   e lhbad...@syr.edu w 
its.syr.edu
SYRACUSE UNIVERSITY
syr.edu



** Participation and subscription information for this EDUCAUSE 
Constituent Group discussion list can be found at 
http://www.educause.edu/discuss.
** Participation and subscription information for this EDUCAUSE 
Constituent Group discussion list can be found at 
http://www.educause.edu/discuss.

**
Participation and subscription information for this EDUCAUSE Constituent Group 
discussion list can be found at http://www.educause.edu/discuss.

RE: Move In/Opening Week- Any Problems?

[Hector J Rios]

Here's ours:

2 8540s in HA mode (bought with the idea of replacing all WiSM2s)
4 pairs of WiSM2s in HA mode
3 server ClearPass cluster for both eduroam and guest
Main SSID: eduroam with PEAP/MSCHAP
Mix of WAPs; 3500, 3600, 3700, 2800, 1810w
Total number of WAPs: 3500
21000 peak users

We tested the 8540s extensively over the spring and summer, primarily with the 
8.2.151 code and a mix of 2800s and 1810ws. We had AVC turned on, and were 
using RLANs for the wired ports. The largest number of WAPs we had on this pair 
was 469. We tested code 8.2.160 towards the end of the summer with all WAPs on 
the 8540s, and had no issues. First the day of classes, we had all WiSM2s 
running 8.2.160 simply as a backup. Early morning we started getting reports of 
802.1X authentication failures (these failures had nothing to do with 
ClearPass). Shortly after that, WAPs starting flapping (disconnecting from the 
8540s moving to WiSM2s and then moving back again). We tried playing with the 
TCP MSS setting, adjusting EAP timers, turning AVC off and multiple other 
things, but nothing worked. In the end, we downgraded the WiSM2s to 8.0.140 and 
moved all WAPs that were not 2800 or 1810s. The 8540s were downgraded to 
8.2.151 so the 2800s and 1810s would have a controller to connect to. Network 
stability was restored after this.

Needless to say it was a very unpleasant experience. We are still working with 
Cisco to find out the root cause of the problem.

Hector Rios
Louisiana State University

From: The EDUCAUSE Wireless Issues Constituent Group Listserv 
[mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU] On Behalf Of Lee H Badman
Sent: Friday, August 25, 2017 8:22 AM
To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU
Subject: [WIRELESS-LAN] Move In/Opening Week- Any Problems?

It might be beneficial to share notes in case other schools are hitting common 
problems. I'm wondering how everyone who is in the thick of it is faring with 
back-to-school?

On this end, we are doing OK halfway to our expected total daily peak clients 
(we're at 15K now high water mark).

Our significant WLAN-related changes since end of Spring semester
* Running 8.2.151 on our 8540s
* Significant quantities of Wave 2 APs
* ISE as RADIUS (only, no NAC, no onboarding)

No changes to:
* our guest WLAN (Clearpass/an Aruba controller pair)
* onboarding (Cloudpath Wiz)
* overall topology
* open network in dorms for gadgets
* non-use of AVC, it crapped out and never got solved after hundreds of 
hours with TAC

Fears:
* We haven't yet hit the scale that will reveal problems with any of 
the newer stuff listed above

Anyone else care to share?

-Lee


Lee Badman | Network Architect

Certified Wireless Network Expert (#200)
Information Technology Services
206 Machinery Hall
120 Smith Drive
Syracuse, New York 13244
t 315.443.3003   f 315.443.4325   e lhbad...@syr.edu w 
its.syr.edu
SYRACUSE UNIVERSITY
syr.edu



** Participation and subscription information for this EDUCAUSE 
Constituent Group discussion list can be found at 
http://www.educause.edu/discuss.

**
Participation and subscription information for this EDUCAUSE Constituent Group 
discussion list can be found at http://www.educause.edu/discuss.

RE: Move In/Opening Week- Any Problems?

[Bucklaew, Jerry]

To ALL:


  I think it might be a bit early to report in, our students come back this 
weekend.  This time of year is normally like a "frog in the pot", every day for 
the next two weeks our counts just keep slowly getting higher and higher.  We 
will see where it ends.



We are an aruba shop with roughly 6,000 access points, we are peaking about 
15,000 clients.  We would expect about 30k so about half way there.  Aruba 
controller code 6.5.2.1, we also have airwave, and clearpass.

This year we completed our Dorm AP install and have about 3,000 205h and 303h 
installed, roughly every other room.  We also have the wired side all connected 
through the controllers to provide the same "experience".   We started 
converting the campus wired to 802.1x or mac  auth also with about 10 buildings 
done.   We have eduroam, our own 802.1x and a mac auth SSID.  We support guest 
and even allow facebook logins.

We have seen two issues so far,



1.   We have seen the aruba 3xx bug where 5ghz utilization is high if you 
have arm scanning enabled.  We disabled Arm scanning for now.

2.   We have seen clearpass deadlock issues (FDB and replication errors) 
based on the endpoint updates we do.  We have spent all week working on 
removing unnecessary updates to see if it will help.  So far so good.

**
Participation and subscription information for this EDUCAUSE Constituent Group 
discussion list can be found at http://www.educause.edu/discuss.

RE: Move In/Opening Week- Any Problems?

[Gray, Sean]

We are still a week away from our students returning, so we have nothing to 
report yet, but here is our setup

Recently upgraded to 8.2.160 on a pair of 5520s (HA)
Mix of APs (totaling just over 600) - 1142,  1532, 1572, 2702, 3602, 3702 & 702w
ACS as RADIUS, soon to be replaced with ISE 2.3

Thanks

Sean


Sean Gray | B.Sc (Hons)
Voice, Collaboration & Wireless Network Analyst
ITS, University of Lethbridge


From: The EDUCAUSE Wireless Issues Constituent Group Listserv 
[mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU] On Behalf Of Lee H Badman
Sent: August-25-17 7:22 AM
To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU
Subject: [WIRELESS-LAN] Move In/Opening Week- Any Problems?

It might be beneficial to share notes in case other schools are hitting common 
problems. I'm wondering how everyone who is in the thick of it is faring with 
back-to-school?

On this end, we are doing OK halfway to our expected total daily peak clients 
(we're at 15K now high water mark).

Our significant WLAN-related changes since end of Spring semester
* Running 8.2.151 on our 8540s
* Significant quantities of Wave 2 APs
* ISE as RADIUS (only, no NAC, no onboarding)

No changes to:
* our guest WLAN (Clearpass/an Aruba controller pair)
* onboarding (Cloudpath Wiz)
* overall topology
* open network in dorms for gadgets
* non-use of AVC, it crapped out and never got solved after hundreds of 
hours with TAC

Fears:
* We haven't yet hit the scale that will reveal problems with any of 
the newer stuff listed above

Anyone else care to share?

-Lee


Lee Badman | Network Architect

Certified Wireless Network Expert (#200)
Information Technology Services
206 Machinery Hall
120 Smith Drive
Syracuse, New York 13244
t 315.443.3003   f 315.443.4325   e lhbad...@syr.edu w 
its.syr.edu
SYRACUSE UNIVERSITY
syr.edu



** Participation and subscription information for this EDUCAUSE 
Constituent Group discussion list can be found at 
http://www.educause.edu/discuss.

**
Participation and subscription information for this EDUCAUSE Constituent Group 
discussion list can be found at http://www.educause.edu/discuss.

RE: Move In/Opening Week- Any Problems?

[Ian Lyons]

Rollins is also seeing about 20% less connected users than this time last year. 
 Same enrollment etc

Waiting for that shoe to drop

From: The EDUCAUSE Wireless Issues Constituent Group Listserv 
[mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU] On Behalf Of Sweetser, Frank E
Sent: Friday, August 25, 2017 12:36 PM
To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU
Subject: Re: [WIRELESS-LAN] Move In/Opening Week- Any Problems?




We were able to complete a couple of major changes in advance of our students 
moving in.



The first is that we completed an upgrade to Aruba 8.1 (brave, I know).  We 
have just under 1,300 APs running on it, mostly 2xx series with a handful of 
3xx.  We had to upgrade our controllers to a pair of HW-MM-5K mobility masters 
(we didn't have a suitable VMware environment to even try the virtual 
controllers), and a pair of 7240XM local controllers.  We had original 7240 
controllers at first, but we found that memory utilization jumped from 60% 
under 6.4, to > 90% in 8.1, so we went with the controller upgrade option as a 
low cost, quick turnaround fix.  We've had some minor glitches on the way, 
especially in the initial setup, but overall has been pretty solid, and hasn't 
caused any user complaints.



Our other major change was to create a new open wireless network that rolled up 
our previous onboarding and guest functionality, and also added in IoT 
capabilities, all backed by Clearpass.  It took quite a bit of work to 
straighten out the authentication rules, even after having a vendor come in and 
help us set up the initial framework.  We've seen lower uptake on the open 
network that we had expected - only around 200 out of 6,000 clients - but our 
best guess is that it's because we still provide on wired port per pillow, 
reducing the need for wireless on gaming and media devices.



Honestly, though, our biggest question is why our peak connected users has 
actually gone down from this time last year.  Previous we were holding steady a 
over 8,000 peak clients, but we haven't come anywhere near that yet.  We're 
waiting to see if people just haven't gotten around to finish registering their 
devices, or if there's some other factor that just hasn't surfaced yet.


Frank Sweetser
Director of Network Operations
Worcester Polytechnic Institute
"For every problem, there is a solution that is simple, elegant, and wrong." - 
HL Mencken


From: The EDUCAUSE Wireless Issues Constituent Group Listserv 
mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU>> 
on behalf of Lee H Badman mailto:lhbad...@syr.edu>>
Sent: Friday, August 25, 2017 9:22 AM
To: 
WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU
Subject: [WIRELESS-LAN] Move In/Opening Week- Any Problems?

It might be beneficial to share notes in case other schools are hitting common 
problems. I'm wondering how everyone who is in the thick of it is faring with 
back-to-school?

On this end, we are doing OK halfway to our expected total daily peak clients 
(we're at 15K now high water mark).

Our significant WLAN-related changes since end of Spring semester
* Running 8.2.151 on our 8540s
* Significant quantities of Wave 2 APs
* ISE as RADIUS (only, no NAC, no onboarding)

No changes to:
* our guest WLAN (Clearpass/an Aruba controller pair)
* onboarding (Cloudpath Wiz)
* overall topology
* open network in dorms for gadgets
* non-use of AVC, it crapped out and never got solved after hundreds of 
hours with TAC

Fears:
* We haven't yet hit the scale that will reveal problems with any of 
the newer stuff listed above

Anyone else care to share?

-Lee


Lee Badman | Network Architect

Certified Wireless Network Expert (#200)
Information Technology Services
206 Machinery Hall
120 Smith Drive
Syracuse, New York 13244
t 315.443.3003   f 315.443.4325   e lhbad...@syr.edu w 
its.syr.edu
SYRACUSE UNIVERSITY
syr.edu



** Participation and subscription information for this EDUCAUSE 
Constituent Group discussion list can be found at 
http://www.educause.edu/discuss.
** Participation and subscription information for this EDUCAUSE 
Constituent Group discussion list can be found at 
http://www.educause.edu/discuss.

**
Participation and subscription information for this EDUCAUSE Constituent Group 
discussion list can be found at http://www.educause.edu/discuss.

Re: Move In/Opening Week- Any Problems?

[Sweetser, Frank E]

We were able to complete a couple of major changes in advance of our students 
moving in.


The first is that we completed an upgrade to Aruba 8.1 (brave, I know).  We 
have just under 1,300 APs running on it, mostly 2xx series with a handful of 
3xx.  We had to upgrade our controllers to a pair of HW-MM-5K mobility masters 
(we didn't have a suitable VMware environment to even try the virtual 
controllers), and a pair of 7240XM local controllers.  We had original 7240 
controllers at first, but we found that memory utilization jumped from 60% 
under 6.4, to > 90% in 8.1, so we went with the controller upgrade option as a 
low cost, quick turnaround fix.  We've had some minor glitches on the way, 
especially in the initial setup, but overall has been pretty solid, and hasn't 
caused any user complaints.


Our other major change was to create a new open wireless network that rolled up 
our previous onboarding and guest functionality, and also added in IoT 
capabilities, all backed by Clearpass.  It took quite a bit of work to 
straighten out the authentication rules, even after having a vendor come in and 
help us set up the initial framework.  We've seen lower uptake on the open 
network that we had expected - only around 200 out of 6,000 clients - but our 
best guess is that it's because we still provide on wired port per pillow, 
reducing the need for wireless on gaming and media devices.


Honestly, though, our biggest question is why our peak connected users has 
actually gone down from this time last year.  Previous we were holding steady a 
over 8,000 peak clients, but we haven't come anywhere near that yet.  We're 
waiting to see if people just haven't gotten around to finish registering their 
devices, or if there's some other factor that just hasn't surfaced yet.


Frank Sweetser
Director of Network Operations
Worcester Polytechnic Institute
"For every problem, there is a solution that is simple, elegant, and wrong." - 
HL Mencken



From: The EDUCAUSE Wireless Issues Constituent Group Listserv 
 on behalf of Lee H Badman 

Sent: Friday, August 25, 2017 9:22 AM
To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU
Subject: [WIRELESS-LAN] Move In/Opening Week- Any Problems?

It might be beneficial to share notes in case other schools are hitting common 
problems. I’m wondering how everyone who is in the thick of it is faring with 
back-to-school?

On this end, we are doing OK halfway to our expected total daily peak clients 
(we’re at 15K now high water mark).

Our significant WLAN-related changes since end of Spring semester

  *   Running 8.2.151 on our 8540s
  *   Significant quantities of Wave 2 APs
  *   ISE as RADIUS (only, no NAC, no onboarding)


No changes to:

  *   our guest WLAN (Clearpass/an Aruba controller pair)
  *   onboarding (Cloudpath Wiz)
  *   overall topology
  *   open network in dorms for gadgets
  *   non-use of AVC, it crapped out and never got solved after hundreds of 
hours with TAC


Fears:

  *   We haven’t yet hit the scale that will reveal problems with any of the 
newer stuff listed above


Anyone else care to share?

-Lee


Lee Badman | Network Architect

Certified Wireless Network Expert (#200)
Information Technology Services
206 Machinery Hall
120 Smith Drive
Syracuse, New York 13244
t 315.443.3003   f 315.443.4325   e lhbad...@syr.edu w 
its.syr.edu
SYRACUSE UNIVERSITY
syr.edu



** Participation and subscription information for this EDUCAUSE 
Constituent Group discussion list can be found at 
http://www.educause.edu/discuss.

**
Participation and subscription information for this EDUCAUSE Constituent Group 
discussion list can be found at http://www.educause.edu/discuss.

RE: Move In/Opening Week- Any Problems?

[ADAM T FERRERO]

Lee and friends,

  We changed several things and are faring very well this move in.  With just 6 
thousand clients in residence halls we aren't at half way yet.  We top around 
12 - 14k devices there which should hit before the weekend ends.


-  Aruba wireless with new controller code this year (6.5.3.2).  Just 
about one AP per suite same as previous years.

-  WPA2 enterprise with posture checking and onboarding via Aruba 
Clearpass (now at 6.6.5)

o   Big changes here, we used to require our own Symantec endpoint so the 
onboarding process was captive portal them to feed the Aruba Onguard agent for 
posture checking and then captive portal to feed them Symantec installer.  Now 
we are allowing a larger list of AV products including Windows Defender and 
Mac's built in firewall / AV.  So, the onboarding is reduced a step and the 
calls for help is very low this year.  Onguard will even auto remediate and 
enable Defender or builtin  firewall if nothing else exists.

-  WPA2 PSK for other devices still in effect.  They pre-register the 
MAC addresses via Clearpass portal.

-  Wired 802.1x no change.  Still barely anyone connecting devices (~3% 
wired vs wireless).  We reviewed the what if we were 100% wireless and we still 
needed 95% of the wired switches just to feed those wireless APs.  So we keep 
offering it as an option.

-  No open wireless except for onboarding guests (also Clearpass) but 
that's everywhere not just Residence Halls.

-  No multicast support so printers and IoT things don't usually work.  
We are working on using Clearpass to limit visibility of those devices just for 
the users that own them.  Aruba has all this functionality available.  We are 
going to test in coming months so we can enable it at a later date.  We've 
purposefully left it off for now not wanting the whole of our enterprise to see 
and print to some poor students printer.

  Good luck with start of school everyone!

  Adam


From: The EDUCAUSE Wireless Issues Constituent Group Listserv 
[mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU] On Behalf Of Lee H Badman
Sent: Friday, August 25, 2017 9:22 AM
To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU
Subject: [WIRELESS-LAN] Move In/Opening Week- Any Problems?

It might be beneficial to share notes in case other schools are hitting common 
problems. I'm wondering how everyone who is in the thick of it is faring with 
back-to-school?

On this end, we are doing OK halfway to our expected total daily peak clients 
(we're at 15K now high water mark).

Our significant WLAN-related changes since end of Spring semester
* Running 8.2.151 on our 8540s
* Significant quantities of Wave 2 APs
* ISE as RADIUS (only, no NAC, no onboarding)

No changes to:
* our guest WLAN (Clearpass/an Aruba controller pair)
* onboarding (Cloudpath Wiz)
* overall topology
* open network in dorms for gadgets
* non-use of AVC, it crapped out and never got solved after hundreds of 
hours with TAC

Fears:
* We haven't yet hit the scale that will reveal problems with any of 
the newer stuff listed above

Anyone else care to share?

-Lee


Lee Badman | Network Architect

Certified Wireless Network Expert (#200)
Information Technology Services
206 Machinery Hall
120 Smith Drive
Syracuse, New York 13244
t 315.443.3003   f 315.443.4325   e lhbad...@syr.edu w 
its.syr.edu
SYRACUSE UNIVERSITY
syr.edu



** Participation and subscription information for this EDUCAUSE 
Constituent Group discussion list can be found at 
http://www.educause.edu/discuss.

**
Participation and subscription information for this EDUCAUSE Constituent Group 
discussion list can be found at http://www.educause.edu/discuss.