Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Damian Menscher via Outages
For future reference: https://developers.google.com/speed/public-dns/cache will let you request a cache flush. Damian On Tue, Oct 7, 2014 at 2:46 PM, John Barbieri via Outages < outages@outages.org> wrote: > In the process of moving, these commands were all run within 5 seconds. > > > jbarbieri

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread John Barbieri via Outages
In the process of moving, these commands were all run within 5 seconds. jbarbieri@jbarbieri-asus:~$ host heartbeat.belkin.com 8.8.8.8 Using domain server: Name: 8.8.8.8 Address: 8.8.8.8#53 Aliases: heartbeat.belkin.com has address 54.163.74.132 heartbeat.belkin.com has address 54.163.87.225 hea

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Jeremy Chadwick via Outages
It looks to me, when querying Belkin's setup directly, they are returning RR A records. (Please pay attention to the "QUESTION SECTION" below, as it's going to become relevant): $ dig @a.gtld-servers.net ns belkin.com. ;; QUESTION SECTION: ;belkin.com.IN NS ;; AUTHORITY

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Michael Loftis via Outages
It's not just the local. 8.8.8.8 is caching the old entry still. Check the authoritative resolvers. On Tue, Oct 7, 2014 at 2:14 PM, Grant Ridder via Outages wrote: > Your large host list looks like it is just your local resolver. > > host heartbeat.belkin.com 8.8.8.8 > > Using domain server: > >

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Luke Rockwell via Outages
http://tcrn.ch/1y30B6H On Tue, Oct 7, 2014 at 2:23 PM, Chris Adams via Outages wrote: > Once upon a time, Grant Ridder via Outages said: > > Your large host list looks like it is just your local resolver. > > DNS records have been changed; it is now hosted on Amazon. > > $ dig +trace heartbea

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Frank Bulk via Outages
Most of our resolvers show that same long list. I assume in a few hours they all will, once the TTL expires. Frank From: Outages [mailto:outages-boun...@outages.org] On Behalf Of Grant Ridder via Outages Sent: Tuesday, October 07, 2014 4:15 PM To: Paul Miller Cc: outages@outages.org Subj

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Chris Adams via Outages
Once upon a time, Grant Ridder via Outages said: > Your large host list looks like it is just your local resolver. DNS records have been changed; it is now hosted on Amazon. $ dig +trace heartbeat.belkin.com ; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.23.rc1.el6_5.1 <<>> @c.root-servers.net +trace hear

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Grant Ridder via Outages
Your large host list looks like it is just your local resolver. host heartbeat.belkin.com 8.8.8.8 Using domain server: Name: 8.8.8.8 Address: 8.8.8.8#53 Aliases: heartbeat.belkin.com has address 67.20.176.130 On Tue, Oct 7, 2014 at 1:48 PM, Paul Miller via Outages wrote: > On Tue, Oct 07,

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Frank Bulk via Outages
I wish they added 127.0.0.1 to that list. =) Frank -Original Message- From: Outages [mailto:outages-boun...@outages.org] On Behalf Of Paul Miller via Outages Sent: Tuesday, October 07, 2014 3:48 PM To: outages@outages.org Subject: Re: [outages] Belkin [heartbeat.belkin.com] On Tue, Oct

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Paul Miller via Outages
On Tue, Oct 07, 2014 at 02:43:13PM -0400, Nick Olsen via Outages wrote: > The current thought (Around the NOC here) is that DNS is breaking due to > another user friendly feature. Where the router forwards you to it's config > page automatically, if it thinks you have no internet.. I still don'

Re: [outages] IPv6 routing loop in GTT ?

2014-10-07 Thread Eric via Outages
I had v6 routing loop issues with GTT as well and so did another person that i know. The provider appears to have fixed it by doing a BGP pref change to Level3 or something On 10/7/2014 11:52 AM, Mark Felder via Outages wrote: On Tue, Oct 7, 2014, at 13:16, Adam Davenport via Outages wrote:

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Carsten Bormann via Outages
On 07 Oct 2014, at 19:57, Frank Bulk via Outages wrote: > Why do that if Belkin is up and running again? This thing seems to go down and up like a yoyo. (I’m seeing it one hop behind 66.179.229.150 — works this very moment.) Can’t get a reliable OS fingerprint (unless it’s an Airport Extreme…).

Re: [outages] IPv6 routing loop in GTT ?

2014-10-07 Thread Mark Felder via Outages
On Tue, Oct 7, 2014, at 13:16, Adam Davenport via Outages wrote: > Mark, > > Give this a second look now if you could. If you're still experiencing > any issues, feel free to ping me directly off-list. Thanks! > Working as expected now. Thanks! ___ O

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread John Neiberger via Outages
Frank, I'd be very interested to hear if that helps. We're considering doing the same thing since a couple of people have reported good results. John On Tue, Oct 7, 2014 at 12:04 PM, Frank Bulk wrote: > No, we’re still getting customer reports about the light staying orange. I > was just hopin

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Nick Olsen via Outages
In my experience. It breaks DNS Masquerading. So any DNS query sent to the router goes unanswered. Verified this with a customer by having them hard-set their Local DNS (On their computer) to my public resolvers. I suspect anything that didn't require DNS worked fine (RDP inbound..SIP Outbound

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Roland Dobbins via Outages
On Oct 8, 2014, at 12:57 AM, Frank Bulk via Outages wrote: > Why do that if Belkin is up and running again? So that it doesn't recur if things get dorked up, again? It's a great DDoS vector - take down that IP, wedge every affected Belkin CPE on the Internet. This kind of thing is extremely

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread John Neiberger via Outages
Just saw another report that HTTPS and IRC were still working. It looks like this was only port 80 that was busted, so I think we can safely add web filtering or parental controls somewhere into the chain of busted things today. On Tue, Oct 7, 2014 at 12:14 PM, Josh Luthman wrote: > Doesn't appe

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Josh Luthman via Outages
Doesn't appear to be all running, at least their official posting doesn't say so - https://belkininternationalinc.statuspage.io/ Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 On Tue, Oct 7, 2014 at 1:57 PM, Frank Bulk via Outages wrote: > Why

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Frank Bulk via Outages
No, we’re still getting customer reports about the light staying orange. I was just hoping they were having other issues. I’ve just added 67.20.176.130 as a loopback in our network, too. Frank From: John Neiberger [mailto:jneiber...@gmail.com] Sent: Tuesday, October 07, 2014 1:01 PM

Re: [outages] IPv6 routing loop in GTT ?

2014-10-07 Thread Adam Davenport via Outages
Mark, Give this a second look now if you could. If you're still experiencing any issues, feel free to ping me directly off-list. Thanks! On 10/7/14, 8:55 AM, Mark Felder via Outages wrote: I am using Charter's 6rd IPv6 tunnel at home and noticed within the last few days I can't get to/from m

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread John Neiberger via Outages
I just saw a report from another ISP who says they had a customer whose VoIP continued to work and they were able to RDP into their computer from the outside. It appears that only web traffic was broken. I wonder if parental controls or web filtering are a component in this fiasco. John On Tue, O

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread John Neiberger via Outages
Given the number of current social media postings, it doesn't appear that they are back up yet, but I suppose it is hard to say for sure given the number of people affected. Twitter is all aflutter with new reports of people still having problems. Yours is the only report that indicated they were w

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Josh Luthman via Outages
I've had a few companies tell me it fixed their problems. A friend that runs a call center said volume was way down an hour or two ago. Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 On Tue, Oct 7, 2014 at 1:42 PM, John Neiberger via Outages < ou

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Frank Bulk via Outages
Why do that if Belkin is up and running again? Or are you suggesting that Belkin doesn’t have it all worked out? Frank From: Outages [mailto:outages-boun...@outages.org] On Behalf Of John Neiberger via Outages Sent: Tuesday, October 07, 2014 12:42 PM To: outages@outages.org Subject: Re: [

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread John Neiberger via Outages
Has anyone else tried the loopback/heartbeat workaround with success? I've seen a few reports so far that this has temporarily solved the problem. John On Tue, Oct 7, 2014 at 10:28 AM, Frank Bulk wrote: > I can ping heartbeat.belkin.com and the DSL Reports link and other > connected links are s

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread John Neiberger via Outages
I just saw a forum post where Belkin recommended manually setting DNS to see if that solves the problem. Putting all of this together, this is my best guess as to what happened. 1. Routers regularly send heartbeat messages to heartbeat.belkin.com 2. Some routing failure causes quite a few netblock

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Frank Bulk via Outages
I can ping heartbeat.belkin.com and the DSL Reports link and other connected links are stating it was back up roughly 30 to 45 minutes ago, so I think we’re out of the woods. Frank From: Outages [mailto:outages-boun...@outages.org] On Behalf Of John Neiberger via Outages Sent: Tuesday, O

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread John Barbieri via Outages
DSLR: http://www.dslreports.com/forum/r29585798-Belkin-Router-Owners-Take-Note. And I am thinking the same thing you are. Doubt resi users upgraded their router overnight. Like you said, chances are it has to do with that heartbeat. Asked a few folks around the US to ping it, some successful

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread John Neiberger via Outages
I think it was initially suspected to be a firmware upgrade, but I doubt that's the case now. It's interesting to me that there are two seemingly unrelated fixes: 1. Put the IP for heartbeat.belkin.com on a loopback 2. Manually add working DNS entries to affected PCs Reports indicate that either

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Josh Luthman via Outages
I believe it needs to both be pingable and attempt an HTTP request. If the HTTP request fails and ping works, the router just assumes the Internet is working. If it can't ping it, it assumes the "Internet is down". I don't have any Belkin routers on hand so I can't confirm this theory. Josh L

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Frank Bulk via Outages
Please provide more information on this – our helpdesk is getting calls about this. Are you saying just route 67.20.176.130 to a pingable IP? Frank From: Outages [mailto:outages-boun...@outages.org] On Behalf Of Josh Luthman via Outages Sent: Tuesday, October 07, 2014 10:42 AM To: Paul M

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Josh Luthman via Outages
Where? I keep reading "bad firmware" problems but everything suggests there is no automatic upgrade and I think we can safely assume resi users didn't upgrade their routers over night. Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 On Tue, Oct 7,

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Josh Luthman via Outages
Just because you can ping it does not mean it will work. I've confirmed with other users this is the case. I believe there is some sort of HTTP request that confirms the connection is working (like Windows, Android, etc). Another resolution is to just loopback 67.20.176.130 (seems to be this IP

Re: [outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread John Barbieri via Outages
Funny, Just saw something about Belkin routers and a botched firmware upgrade. I wonder if this is what they meant. That being said, it is working fine from Zito Media in Clanton, AL. --John On 10/07/2014 10:17 AM, Paul Miller via Outages wrote: On one of my netblocks my subscribers can nolon

[outages] Belkin [heartbeat.belkin.com]

2014-10-07 Thread Paul Miller via Outages
On one of my netblocks my subscribers can nolonger ping Belkin. I can ping heartbeat.belkin.com from my webservers and from subscriber units on a different netblock. Apparently Belkin routers ping the home base every couple seconds to determine the correct color of the Internet light on the route

[outages] IPv6 routing loop in GTT ?

2014-10-07 Thread Mark Felder via Outages
I am using Charter's 6rd IPv6 tunnel at home and noticed within the last few days I can't get to/from my Digital Ocean VM via IPv6. From both Digital Ocean and Charter 6rd tunnel I can get to Google, Facebook, etc via IPv6 but not to each other. It appears there are routing issues in GTT as it hit