Matthew, We experienced a similar dynamic when we first launched 802.11n with 802.1x. We moved from a previous system that was *completely* open (by necessity, not desire) to a secure auth system (very desired).
There was certainly a social media backlash as the new authentication was painful by comparison. Peer organizations we spoke with on the matter said, to paraphrase, “Why are you paying attention to Twitter?” Not that this is the advice you want, but we eventually proved that based on volume of successful authentications and volume of peak concurrent devices that the social media complains amounted to about 0.002% of authentications and 0.4% of the user base. We’re not on the same system unfortunately so I can’t offer exact steps to remedy. We’re using Juniper (formerly Trapeze) and we are able to run weekly reports of our 802.1x auth failures, assoc. failures, etc. They’re not entirely helpful - most failures occur with improper MSCHAPv2 credentials so we see a failure for “John’s PC,” not jdoe4...@college.edu<mailto:jdoe4...@college.edu>. Our system does log MAC address, but we don’t require our BYOD users to register the MAC address of their machine with the school, so we miss 1:1 correlation in that department as well. But we did find about 25% of the data collected in the early running helped us track down valid credentials and assist users. As to quality metrics…we received that one as well. We’ve only recently completed super-gluing one-another’s heads back together. What we opted for was to: 1) graph and measure bandwidth used by our wireless networks: this accounts for roughly 2/3 of the bandwidth used on campus or stated conversely is approximately 2 times the bandwidth used on the “wired” networks; 2) count and chart successful authentications by month*; 3) graph session load on each wireless controller via Cacti and create an aggregate graph showing total sessions at 5 minute intervals with min, max and averages output. * A member of our access management team was kind enough to provide us data with time stamps for successful authentications so we were able to demonstrate authentications per minute. I think we had several peaks in the mid-thousands per minute. I would have to look back at the data. But, short story longer, we were able to provide enough data that essentially 10^7th auths per month for a student body of 10^4 students with peak concurrent sessions of 10^4 / 2 and bandwidth of ‘x’ Gbps = quality user experience. It is as close as we were able to get to assessing end user experience. With our system, we can track the performance of individual sessions, average SNR, RSSI and duration, but the numbers were more difficult to work with than we anticipated. Hope this was mildly helpful. - Scot On Oct 22, 2015, at 8:11 AM, Williams, Matthew <mwill...@kent.edu<mailto:mwill...@kent.edu>> wrote: Thank you for the ideas, everyone. The problem that we have with measuring tickets is that our user base is more apt to complain on social media and we simply don’t have the man-hours to scour the various sites. Another component that we were thinking about trying to tie into this metric is disconnects, but I can’t find a reliable report from Cisco Prime Infrastructure. At any rate, I’ll start with all of the great feedback from all of you and see where we end up. Thanks again! Respectfully, Matthew Williams Manager, Network and Telecommunications Services Kent State University Office: (330) 672-7246 Mobile: (330) 469-0445 From: The EDUCAUSE Wireless Issues Constituent Group Listserv [mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU] On Behalf Of Pete Hoffswell Sent: Thursday, October 22, 2015 8:52 AM To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU<mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU> Subject: Re: [WIRELESS-LAN] Measuring User Experience I pretty much measure this by number of tickets opened for wifi issues at our support desk. It's a pretty good spot to measure "user experience". Perhaps the only spot. - Pete Hoffswell - Network Manager pete.hoffsw...@davenport.edu<mailto:pete.hoffsw...@davenport.edu> http://www.davenport.edu<http://www.davenport.edu/> On Thu, Oct 22, 2015 at 8:46 AM, Matthew McFall <mmcf...@jsu.edu<mailto:mmcf...@jsu.edu>> wrote: Matthew, Another suggestion would be to do a poll/survey of the end users. Depending on the response you get, you may get some useful feedback. Matthew S. McFall Network Engineer CCNA/CCNA-Security Division of Information Technology Jacksonville State University Office: 256-782-5664<tel:256-782-5664> [http://www.jsu.edu/gem_images/jsu_it_footer.gif] ________________________________ From: "Philippe Hanset" <phan...@anyroam.net<mailto:phan...@anyroam.net>> To: WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU<mailto:WIRELESS-LAN@LISTSERV.EDUCAUSE.EDU> Sent: Thursday, October 22, 2015 7:43:31 AM Subject: Re: [WIRELESS-LAN] Measuring User Experience Matthew, Here are a few ideas (assuming that your Wi-Fi system/Trouble Ticket system allows you to poll that kind of data) -HELP DESK: % of Help Desk trouble tickets related to Wireless, and type of problem -BANDWIDTH: How is your bandwidth to the Internet doing? (% Utilization, are you limited for the Wi-Fi side?) -AP LOAD: What is your campus wide ratio of user/AP (theoretical campus ratio : maximum unique users per day/total number of APs, Distribution ratio: What do you observe in your system, and how many APs are not within your own requirements…) -AP DENSITY: What is your average dBm ? what do you consider to be your requirement and what % of users on APs are/are not within that limit -AP QUALITY: Then move to Jorj suggestion of measuring re-auth etc… Any other limiting factor like DHCP capacity (number of IPs) and finally, ease of configuration (number of trouble tickets related to configuration issues) Philippe Philippe Hanset www.eduroam.us<http://www.eduroam.us/> On Oct 22, 2015, at 7:04 AM, Williams, Matthew <mwill...@kent.edu<mailto:mwill...@kent.edu>> wrote: I have been instructed that I need determine a metric that reasonably guestimates the end user experience of our wireless networks, without procuring a system(s) that does it. I readily admit that my head kind of exploded when this directive was given. Have any of you done this exercise or have any ideas/formulas to try to calculate something like this? Thanks for any ideas that you care to share. Respectfully, Matthew Williams Manager, Network and Telecommunications Services Kent State University Office: (330) 672-7246<tel:%28330%29%20672-7246> Mobile: (330) 469-0445<tel:%28330%29%20469-0445> ********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/. ********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/. ********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/. ********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/. ********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/. ********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/.