Send Outages-discussion mailing list submissions to
outages-discussion@outages.org
To subscribe or unsubscribe via the World Wide Web, visit
https://puck.nether.net/mailman/listinfo/outages-discussion
or, via email, send a message with subject or body 'help' to
outages-discussion-requ...@outages.org
You can reach the person managing the list at
outages-discussion-ow...@outages.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Outages-discussion digest..."
Today's Topics:
1. Re: [outages] Facebook Down (Rusty Dekema)
2. Re: [outages] Facebook Down (Chad Austin)
----------------------------------------------------------------------
Message: 1
Date: Mon, 4 Oct 2021 19:11:21 -0400
From: Rusty Dekema <rdek...@gmail.com>
To: Nate <nat...@gmail.com>
Cc: Matt Hoppes <mattli...@rivervalleyinternet.net>,
outages-discussion <outages-discussion@outages.org>
Subject: Re: [Outages-discussion] [outages] Facebook Down
Message-ID:
<cajxvgblssdbp3pnf6lvh1snccebplqsqqiltqo5ejzlsc_k...@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Probably quite a few, between usual human behavioral oddities and the
fact that the listserv appears to be delivering messages extremely
slowly today. At many times throughout the day, there have been 3-5x
more messages on the outages list web archive on this topic than had
actually arrived in my inbox.
-Rusty
On Mon, Oct 4, 2021 at 5:47 PM Nate <nat...@gmail.com> wrote:
>
> Wonder how many more people are going to say it's down for them, in a certain
> state or country without reading this email chain?
>
> To everyone who DOESNT READ the chain.
> YES!!! Facebook, IG,Messenger, and WhatsApp are down hard, meaning no one
> can access any of those services in any state or country currently.
>
>
> On Mon, Oct 4, 2021, 3:39 PM Rusty Dekema <rdek...@gmail.com> wrote:
>>
>> Facebook is having DNS issues probably stemming from a network
>> engineering issue:
>>
>> > facebook.com.
>> Server: dns.google
>> Address: 2001:4860:4860::8888
>> *** dns.google can't find facebook.com.: Server failed
>>
>> > facebook.com.
>> Server: dns9.quad9.net
>> Address: 9.9.9.9
>> *** dns9.quad9.net can't find facebook.com.: Server failed
>>
>> https://twitter.com/Benjojo12/status/1445054959209287680
>>
>> Facebook owns Instagram and WhatsApp, so this is probably responsible
>> for those services' outages as well.
>>
>> -Rusty
>>
>> On Mon, Oct 4, 2021 at 12:08 PM Matt Hoppes via Outages
>> <outa...@outages.org> wrote:
>> >
>> > Is anyone else seeing Facebook down? Seems to be offline here in
>> > Pennsylvania.. started with error messages now DNS errors.
>> >
>> > Seeing lots of errors on Instagram, QuickBooks, Whatsapp, Amazon.
>> > _______________________________________________
>> > Outages mailing list
>> > outa...@outages.org
>> > https://puck.nether.net/mailman/listinfo/outages
>> _______________________________________________
>> Outages-discussion mailing list
>> Outages-discussion@outages.org
>> https://puck.nether.net/mailman/listinfo/outages-discussion
------------------------------
Message: 2
Date: Mon, 4 Oct 2021 19:12:39 -0400
From: Chad Austin <m...@chadaustin.org>
To: James Liakos <ja...@liakos.com.au>,
"outages-discussion@outages.org" <outages-discussion@outages.org>
Subject: Re: [Outages-discussion] [outages] Facebook Down
Message-ID: <ebe97006-d7fd-463c-6b9a-747a63b7d...@chadaustin.org>
Content-Type: text/plain; charset="windows-1252"; Format="flowed"
It looks like they learned their lesson, but are now applying it to
reddit comments.
-Chad
On 10/4/2021 7:07 PM, James Liakos wrote:
> Or maybe commit confirmed
> ------------------------------------------------------------------------
> *From:* Outages-discussion <outages-discussion-boun...@outages.org> on
> behalf of Chad Austin <m...@chadaustin.org>
> *Sent:* Tuesday, October 5, 2021 9:39:50 AM
> *To:* outages-discussion@outages.org <outages-discussion@outages.org>
> *Subject:* Re: [Outages-discussion] [outages] Facebook Down
> [EXTERNAL EMAIL]
>
> Someone must have gone a long time before they finally learned the
> importance of the reload-in command.
>
> -Chad
>
> On 10/4/2021 1:28 PM, Matt Hoppes wrote:
> > That's bad... if I read that properly.. they've locked themselves out
> > of their routers and are unable to get in without physically touching
> > each one.
> >
> > On 10/4/21 1:00 PM, Eric Henson wrote:
> >> Someone on the Facebook Incident response team is posting updates to
> >> Reddit.
> >>
> >>
> https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.reddit.com%2Fuser%2Framenporn%2Fcomments%2F&data=04%7C01%7Cjames%40liakos.com.au%7Cc59c6e5114ba4dafc6de08d98789c15b%7C6ad8265a93b14b7990a74df444e5977e%7C0%7C0%7C637689849044900064%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Pv3C6OqnOTsOZYrsBB0mDc%2BkE7T6tOc149SwLvuQzS8%3D&reserved=0
>
> <https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.reddit.com%2Fuser%2Framenporn%2Fcomments%2F&data=04%7C01%7Cjames%40liakos.com.au%7Cc59c6e5114ba4dafc6de08d98789c15b%7C6ad8265a93b14b7990a74df444e5977e%7C0%7C0%7C637689849044900064%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Pv3C6OqnOTsOZYrsBB0mDc%2BkE7T6tOc149SwLvuQzS8%3D&reserved=0>
> >>
> >> ?As many of you know, DNS for FB services has been affected and this
> >> is likely a symptom of the actual issue, and that's that *BGP peering
> >> with Facebook peering routers has gone down*, very likely due to a
> >> configuration change that went into effect shortly before the outages
> >> happened (started roughly 1540 UTC).
> >>
> >> There are people now trying to gain access to the peering routers to
> >> implement fixes, but the people with physical access is separate from
> >> the people with knowledge of how to actually authenticate to the
> >> systems and people who know what to actually do, so there is now a
> >> logistical challenge with getting all that knowledge unified.
> >>
> >> Part of this is also due to lower staffing in data centers due to
> >> pandemic measures.?
> >>
> >> --------
> >>
> >> *Eric Henson*
> >>
> >> Senior Manager, IT Operations
> >>
> >> PFSweb, Inc.
> >>
> >> *o*:? 972.685.1387
> >>
> >> *m:*972.948.3424
> >>
> >>
> https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.pfsweb.com%2F&data=04%7C01%7Cjames%40liakos.com.au%7Cc59c6e5114ba4dafc6de08d98789c15b%7C6ad8265a93b14b7990a74df444e5977e%7C0%7C0%7C637689849044910026%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=qfgtHba0DojZNkvmOF7u2sYjhxGfv3APRWn4k85oCUs%3D&reserved=0
>
> <https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.pfsweb.com%2F&data=04%7C01%7Cjames%40liakos.com.au%7Cc59c6e5114ba4dafc6de08d98789c15b%7C6ad8265a93b14b7990a74df444e5977e%7C0%7C0%7C637689849044910026%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=qfgtHba0DojZNkvmOF7u2sYjhxGfv3APRWn4k85oCUs%3D&reserved=0>
> >>
> >> *From:* Outages <outages-boun...@outages.org> *On Behalf Of *Link IP
> >> via Outages
> >> *Sent:* Monday, October 4, 2021 10:51 AM
> >> *To:* Matt Hoppes <mattli...@rivervalleyinternet.net>
> >> *Cc:* outa...@outages.org
> >> *Subject:* Re: [outages] Facebook Down
> >>
> >> *CAUTION:*This email originated from outside of the organization. Do
> >> not click links or open attachments unless you recognize the sender
> >> and know the content is safe.
> >>
> >> Down here in? the UK as well
> >>
> >> On Mon, 4 Oct 2021 at 16:48, Matt Hoppes via Outages
> >> <outa...@outages.org <mailto:outa...@outages.org
> <mailto:outa...@outages.org>>> wrote:
> >>
> >>???? Is anyone else seeing Facebook down??? Seems to be offline here in
> >>???? Pennsylvania.. started with error messages now DNS errors.
> >>
> >>???? Seeing lots of errors on Instagram, QuickBooks, Whatsapp, Amazon.
> >> _______________________________________________
> >>???? Outages mailing list
> >>???? outa...@outages.org <mailto:outa...@outages.org
> <mailto:outa...@outages.org>>
> >>
> https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpuck.nether.net%2Fmailman%2Flistinfo%2Foutages&data=04%7C01%7Cjames%40liakos.com.au%7Cc59c6e5114ba4dafc6de08d98789c15b%7C6ad8265a93b14b7990a74df444e5977e%7C0%7C0%7C637689849044910026%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4gVfA0Tii2zABZR1ugNztgsYarea8ytRrKKbh7llQJ8%3D&reserved=0
>
> <https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpuck.nether.net%2Fmailman%2Flistinfo%2Foutages&data=04%7C01%7Cjames%40liakos.com.au%7Cc59c6e5114ba4dafc6de08d98789c15b%7C6ad8265a93b14b7990a74df444e5977e%7C0%7C0%7C637689849044910026%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4gVfA0Tii2zABZR1ugNztgsYarea8ytRrKKbh7llQJ8%3D&reserved=0>
> >>
> >>
> >> --
> >>
> >> *Lance Davis*
> >>
> >> *Link IP Networks Limited*
> >> Delta House,
> >> Wavell Road,
> >> Wythenshawe,
> >> M22 5QZ
> >>
> >> Office Tel: 0161 813 1870
> >>
> >> Mobile Tel: 07703 196010
> >>
> >> Email: la...@linkipnetworks.co.uk
> <mailto:la...@linkipnetworks.co.uk <mailto:la...@linkipnetworks.co.uk>>
> >>
> >> Link IP Networks Limited is a company registered in England and Wales
> >> with registration number 11881958 and registered address Delta House,
> >> Wavell Road, Wythenshawe, M22 5QZ.
> >>
> >> This e-mail and any files transmitted with it are confidential and/or
> >> legally privileged and intended solely for the use of the individual
> >> or entity to whom they are addressed. If you are not the addressee
> >> indicated in this e-mail (or responsible for the delivery of the e
> >> mail to such person) be advised that you have received this e-mail in
> >> error, and that any use, dissemination, forwarding, printing, or
> >> copying of this e-mail is strictly prohibited. If you have received
> >> this e-mail in error please notify the sender by reply e-mail and
> >> then immediately permanently delete it and all copies from your
> >> system. Whilst we maintain virus checks, we accept no liability for
> >> viruses introduced from this e-mail.
> >>
> >> ?Please consider the *environment* before printing this e-mail
> >>
> > _______________________________________________
> > Outages-discussion mailing list
> > Outages-discussion@outages.org
> >
> https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpuck.nether.net%2Fmailman%2Flistinfo%2Foutages-discussion&data=04%7C01%7Cjames%40liakos.com.au%7Cc59c6e5114ba4dafc6de08d98789c15b%7C6ad8265a93b14b7990a74df444e5977e%7C0%7C0%7C637689849044910026%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=dlDwjgre0MHXq%2F499A8BS69ICSJSKDiXmwNfXcKUWIU%3D&reserved=0
>
> <https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpuck.nether.net%2Fmailman%2Flistinfo%2Foutages-discussion&data=04%7C01%7Cjames%40liakos.com.au%7Cc59c6e5114ba4dafc6de08d98789c15b%7C6ad8265a93b14b7990a74df444e5977e%7C0%7C0%7C637689849044910026%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=dlDwjgre0MHXq%2F499A8BS69ICSJSKDiXmwNfXcKUWIU%3D&reserved=0>
> _______________________________________________
> Outages-discussion mailing list
> Outages-discussion@outages.org
> https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpuck.nether.net%2Fmailman%2Flistinfo%2Foutages-discussion&data=04%7C01%7Cjames%40liakos.com.au%7Cc59c6e5114ba4dafc6de08d98789c15b%7C6ad8265a93b14b7990a74df444e5977e%7C0%7C0%7C637689849044910026%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=dlDwjgre0MHXq%2F499A8BS69ICSJSKDiXmwNfXcKUWIU%3D&reserved=0
>
> <https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpuck.nether.net%2Fmailman%2Flistinfo%2Foutages-discussion&data=04%7C01%7Cjames%40liakos.com.au%7Cc59c6e5114ba4dafc6de08d98789c15b%7C6ad8265a93b14b7990a74df444e5977e%7C0%7C0%7C637689849044910026%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=dlDwjgre0MHXq%2F499A8BS69ICSJSKDiXmwNfXcKUWIU%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://puck.nether.net/pipermail/outages-discussion/attachments/20211004/0bf6d340/attachment.htm>
------------------------------
Subject: Digest Footer
_______________________________________________
Outages-discussion mailing list
Outages-discussion@outages.org
https://puck.nether.net/mailman/listinfo/outages-discussion
------------------------------
End of Outages-discussion Digest, Vol 140, Issue 6
**************************************************
