Reverting to Windows 7 isn't likely to be good advice. Turning off updates also isn't likely to be good advice.
----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "David" <[email protected]> To: [email protected] Sent: Thursday, July 14, 2016 4:05:01 PM Subject: Re: [AFMUG] CDN overload We sent this the below instructions out to a section of customers who are on a small upstream and were killing on Update Tuesday and the day after. The metering option helps if they have PC's connected via wifi. Otherwise disabling the windows update service helps too :) A big disclaimer *at your own risk* was sent with the instructions.... We had about 7% of all users complain yesterday, and yes, all had windows 10. Most had just been converted over. We also sent instructions on how to revert back to windows 7 if they were in the 30 day period, and how to install a blocker to block windows 7 if anyone was interested. We got a lot of hits on that one and support was busy, but people were really happy for the help. Set Wi-Fi to Metered 1. Connected Make sure you are connected to the Wi-Fi network that you wish to set as metered 2. All Settings 3. Network & Internet 4. Advanced options You may have to scroll down if you have a lot of networks. Assuming you are connected to the network you want to set as metered, hit Advanced options for the next step. 5. Turn on Under Metered connection, you can toggle Set as metered connection to On . That's it. Now that Wi-Fi connection is set to metered, and the OS will throttle data usage for just the barebones like user initiated web browsing or email checking. Turn off Windows Updates in Windows 10 This is the best way to make sure that windows does not preform excessive updates: You can do this using the Windows Update service. Go to Control Panel Click Administrative Tools Double click Services In the Services window, scroll down to Windows Update. To turn it off, right-click on the process, click on Properties and select Disabled. Click OK. Then right click on the Windows update service and Click "Stop" That will take care of Windows Updates not being installed on your machine. On Thu, Jul 14, 2016 at 12:29 PM, Mathew Howard < [email protected] > wrote: It seems like it's both. sometimes they're all to one IP, and sometimes it's a bunch of different IPs... but I guess that could be multiple computers trying to update. I don't think it's just 13.x.x.x sources either, it looks to me like they're using CDN's too - I saw what looked like the same kind of traffic coming from akamai IPs. On Thu, Jul 14, 2016 at 2:50 PM, Adam Moffett < [email protected] > wrote: <blockquote> Are the many connections open to the same address or are they spread among multiple addresses? If they're opening dozens of connections to the same IP, then I think a fairly simple rule could cap the number of connections between any given src and dst pair. Set it to 10 or 15 and it shouldn't break any sane service. If it's multiple src IP's then put a cap on number of connections between 13.x.x.x src and any of your IP's. ------ Original Message ------ From: "Craig Schmaderer" < [email protected] > To: " [email protected] " < [email protected] > Sent: 7/14/2016 3:25:21 PM Subject: Re: [AFMUG] CDN overload <blockquote> Same here, it didn’t take me very long to assume it was windows 10 updates, so the question is, is this going to be the norm? every update Tuesday going to f**k the network up? Does anyone have some good info or idea on what is going to happen? From: Af [mailto: [email protected] ] On Behalf Of Ken Hohhof Sent: Thursday, July 14, 2016 1:26 PM To: [email protected] Subject: Re: [AFMUG] CDN overload Wowzers. I guess I feel better knowing it’s not just me, this really had my head spinning, trying to figure out who was attacking me. But yeah, they changed something this week. It used to be pretty common to see 4 parallel sessions, 93 is just crazy, hard to interpret it any other way than not playing nice and pushing all other traffic aside. But we all know ISPs are evil and Silicon Valley can do no wrong, so it must be our fault somehow. From: Mathew Howard Sent: Thursday, July 14, 2016 10:10 AM To: af Subject: Re: [AFMUG] CDN overload Just had another one call in... 93 active http sessions to 13.107.4.50, Microsoft obviously changed something with how they're doing update... I haven't ever seeing so many complaints generated from an update before. On Thu, Jul 14, 2016 at 8:13 AM, Nate Burke < [email protected] > wrote: <blockquote> I just ran into this yesterday, took down an FSK AP that was running at 10mb Ethernet. A customer with 2 computers, MS Updates running in the background. Had about 50 http sessions open to 13.x.x.x addresses. On 7/14/2016 7:50 AM, Adam Moffett wrote: <blockquote> Seems like they (MS) should look into promoting a multicast network for distributing updates. Or simply limit automatic background updates to 256k (per destination). If the user clicked the update button, sure get it to run as fast as possible, but if it's in the background and they don't even know it's happening then it ought to not matter how long the download takes. ...of course MS is not likely to care about my opinion on the matter. ------ Original Message ------ From: "George Skorup" < [email protected] > To: [email protected] Sent: 7/14/2016 2:33:21 AM Subject: Re: [AFMUG] CDN overload <blockquote> I forgot about this. Yes. A little later in the day, I started to see a lot of 13.n.n.n sources. Microsoft. Yeah, update Tuesday. Then the same customer would start receiving from LLNW. Then Akamai. And back to MS again. So it looks like they're *still* distributing updates across various CDNs. And believe me, it's not like they were all hitting this customer at once. One single CDN would try to send at 5-10X the customer's downlink MIR. Sometimes more. At one point I saw over 20Mbps for 5-10 minutes. I saw pretty much the same thing with about 15 other customers that I looked at. And they were spread across 5-6 towers. Some directly licensed fed, others farther towards the edge. DDoS. CDN. Same thing. Or gorilla tactics at the very least. If the customer calls and says "none of my other shit works, your internet sucks" what are we supposed to do? Oh OK, here, we'll turn you up to 12Mbps and see what that does. Yeah screw that because now the CDN is sending at 40Mbps! They need to stop fucking with TCP already! And no, it doesn't matter where I put the policing/shaping. They still eat up bandwidth on our upstreams. Like you said before Ken, yeah, it just moves the problem somewhere else. On 7/13/2016 11:39 PM, Ken Hohhof wrote: <blockquote> George, did you identify the application or content provider, or only the CDN? I think I started getting hit with the same thing early yesterday afternoon. At first I thought I was getting DDOS attacks. From: George Skorup Sent: Tuesday, July 12, 2016 6:21 PM To: [email protected] Subject: Re: [AFMUG] CDN overload Yup. LLNW. On 7/12/2016 5:35 PM, Ken Hohhof wrote: <blockquote> I assume you torched the traffic and verified it is all coming from a particular CDN, not a random bunch of IPs as would be the case with BT. Since this isn’t your first rodeo. From: George Skorup Sent: Tuesday, July 12, 2016 5:31 PM To: [email protected] Subject: Re: [AFMUG] CDN overload Because they dick with TCP. On 7/12/2016 5:23 PM, Eric Kuhnke wrote: <blockquote> And why is it the fault of the CDN? It could be a customer with a 100-peer bittorrent session downloading 30GB of Ubuntu DVD ISOs. On Tue, Jul 12, 2016 at 3:13 PM, George Skorup < [email protected] > wrote: <blockquote> I have had it with these CDNs sending more traffic than the last mile can handle. Got a customer at 1.5Mbps on 900 FSK and they're sending to her at 15Mbps. Of course the AP reports RF downlink overloaded. </blockquote> </blockquote> </blockquote> </blockquote> </blockquote> </blockquote> </blockquote> </blockquote> </blockquote> -- David Kunat
