[outages] META: What's fit for the main list

I occasionally catch a little crap -- we've lost some subscribers for it over 8 years or more -- about what I consider a fit topic to announce on this main list. It includes outages or major impacts to what some people call "eyeball" services: things like Twitter, Google, and... Facebook. My pri

[outages] Intermedia Outage - Voice services connectivity issues

Event 1855: Voice services connectivity issues Tuesday, 05 October 2021 01:15 (Pacific Standard Time) Accounts Potentially Affected: Service(s) Impacted: Voice Core, HPBX, Envision Title: Voice services connectivity issues Start time: Tuesday, 05 October 2021 01:15 (Pacific Sta

[outages] Mitel Outage East Coast

Our mitel system seems to be out in the East Coast. Seems like a wide spread outage related to Mitel. Thanks, George Ly ___ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

Re: [outages] Backlog cleared

- Original Message - > From: "Jared Mauch via Outages" > Apologies for not meeting expectations. I will refund you a full month > service > fee :-) Damn well better. All seriousness aside, I want to take a moment here to thank Jared for the thankless job of running a bunch of mailin

Re: [outages] facebook outage

Ok, I lied, I’m still awake. I got my first successful Facebook main page load at 23:13 UTC, for an overall duration of 8:33, or 513 minutes. Multiplied by three billion users, that’s 1.54 trillion person-minutes. That’s a tera-lapse! Have we had one of those before?

[outages] Backlog cleared

I wanted to give everyone a quick update (reply-to set to just me, so send your gripes/complaints/whatnot there) The mail backlog is cleared. There were approximately 60k e-mails backlogged to go out to people which is now caught up. I learned about a script called qshape which can tell you wh

Re: [outages] Meta: outages outage

Years ago when part of my job was babysitting a commercial subscription stock market research distribution list, we had to create separate queues for major email providers, etc (gmail, Hotmail, yahoo) so that any one of them wouldn't delay others. We ended up with about two dozen separate queues

Re: [outages] Facebook back up

> On Oct 4, 2021, at 5:54 PM, Ryan McGinnis via Outages > wrote: > > Productivity back down > > Due to the outages list backlog, you will probably receive this sometime > after Thanksgiving “I’m not even supposed to be here today” - outages sysadmin staff :-) - Jared

Re: [outages] Larger problems caused by Facebook DNS outage?

I suspect that Android uses facebook.com to determine internet availability . I had the same issue - mobile phone was saying no internet access on wifi when the access was perfectly fine. On Tue, 5 Oct 2021, 00:34 Sebastian Wiesinger via Outages, < outages@outages.org> wrote: > I've received repo

Re: [outages] facebook outage

> On Oct 5, 2021, at 12:16 AM, Bill Woodcock wrote: > > > >> On Oct 4, 2021, at 11:41 PM, Baldur Norddahl >> wrote: >> >> >> >> man. 4. okt. 2021 23.33 skrev Bill Woodcock : >> >> >>> On Oct 4, 2021, at 11:21 PM, Bill Woodcock wrote: >>> >>> >>> On Oct 4, 2021, at 11:10 PM, Bi

Re: [outages] Facebook

Cloudflare says it's BGP https://blog.cloudflare.com/october-2021-facebook-outage/ On Mon, Oct 4, 2021 at 4:53 PM Julian Panke via Outages wrote: > If you look at the amount of SERVFAIL from around the world it seems to be > save to say that it is a global outage: > https://atlas.ripe.net/measu

Re: [outages] Facebook.com no records?

Cloudflare summed it up nicely: https://blog.cloudflare.com/october-2021-facebook-outage/ Rob++ - Original Message - > From: "Jason Kuehl via Outages" > To: "Jason Kuehl via Outages" > Sent: Monday, October 4, 2021 11:26:52 AM > Subject: Re: [outages] Facebook.com no records? > A carr

Re: [outages] Fwd: massive facebook outage presently

Dns needs the ip addresses to be working. Not in any sense unique ! On Tue, 5 Oct 2021, 01:19 Töma Gavrichenkov via Outages, < outages@outages.org> wrote: > Peace, > > On Mon, Oct 4, 2021, 10:57 PM Bill Woodcock via Outages < > outages@outages.org> wrote: > >> Is it always the DNS? We’ll see. >>

Re: [outages] WhatsApp down?

"Facebook is back online after a massive outage that also took down Instagram, WhatsApp, Messenger, and Oculus - The Verge" https://www.theverge.com/platform/amp/2021/10/4/22708989/instagram-facebook-outage-messenger-whatsapp-error On Oct 4, 2021 7:37 PM, Jeff Shultz via Outages wrote: Since he

Re: [outages] Facebook Down

> On Oct 4, 2021, at 7:23 PM, Paul M - outages via Outages > wrote: > > here in the UK, I can't resolve www.facebook.com nor m.facebook.com, > neither with my own caching resolvers, cloudflare or google DNS > > a.ns.facebook.com - 129.134.30.12 - is unresponsive. > > am testing the others an

[outages] Outages [[meta]]

> On Oct 4, 2021, at 11:21 PM, Bill Woodcock via Outages > wrote: I just received this. > Received: from puck.nether.net ([2001:418:3f4::5]) > by mail.pch.net (Kerio Connect 9.2.7 patch 3) with ESMTPS > Tue, 5 Oct 2021 00:54:36 -0700 > > Received: from puck.nether.net (localhost

Re: [outages] facebook outage

> On Oct 4, 2021, at 11:41 PM, Baldur Norddahl > wrote: > > > > man. 4. okt. 2021 23.33 skrev Bill Woodcock : > > > > On Oct 4, 2021, at 11:21 PM, Bill Woodcock wrote: > > > > > > > >> On Oct 4, 2021, at 11:10 PM, Bill Woodcock wrote: > >> > >> They’re starting to pick themselves back up

Re: [outages] Facebook

Restored now and working here now (after 3-4 hrs of outage). Location: India anurag@desktop ~> dig www.facebook.com a ; <<>> DiG 9.16.1-Ubuntu <<>> www.facebook.com a ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56238 ;; flags: qr rd ra; QUERY: 1, AN

Re: [outages] [EXTERNAL] Re: Facebook.com no records?

I’d be surprised if a single person had that much authority. Good news is there might be some engineering jobs open up at FB From: Outages on behalf of John Quinby via Outages Sent: Monday, October 4, 2021 1:03:11 PM To: outages@outages.org Subject: [EXTERNAL

Re: [outages] Facebook Down

First I love this list, but come on y'all. I've gotten like 50 or 60 notifications about this today. Please take it over to the discussions thread. For the rest of the unsubstantiated. As of 2100UTC Facebook has started resuming communications. Here are some major relevant posts for everything

Re: [outages] WhatsApp down?

Again? Em seg., 4 de out. de 2021 21:58, Nate Howe via Outages escreveu: > Both Facebook and Instagram are out here in RI as well. > > Sent from my OnePlus 7T Pro McLaren > -- > *From:* Outages on behalf of J. Hellenthal > via Outages > *Sent:* Monday, October 4, 20

Re: [outages] Meta: outages outage

> On Oct 5, 2021, at 5:34 AM, Jürgen Botz via Outages > wrote: > > OK, aside from the Facebook outage, and no doubt because of it, we > effectively had an outages outage... messages to the lsit were massively > delayed, causing lots of duplicate reports and general failure to have > any meani

[outages] Meta: outages outage

OK, aside from the Facebook outage, and no doubt because of it, we effectively had an outages outage... messages to the lsit were massively delayed, causing lots of duplicate reports and general failure to have any meaningful confirmations, etc. Why? What happened? Well, outages is hosted on a

Re: [outages] facebook outage

> On Oct 4, 2021, at 11:50 PM, Ryan Brooks wrote: > DNS was a victim in this outage, not the cause. You are absolutely correct. However, people who don’t have this problem avoid having this problem by not putting all their DNS eggs in one basket. And then forgetting where they put the basket

Re: [outages] Facebook

It's never DNS. It's the bonehead that messed up DNS. Original message From: Ryan McGinnis via Outages Date: 10/4/21 6:41 PM (GMT-06:00) To: Outages Subject: Re: [outages] Facebook CAUTION: This email is from OUTSIDE our organization. Please do not open/download any attachmen

Re: [outages] Facebook Down

I guess that's mostly due to the fact that mails are arriving very late and out of order. Mit freundlichen Grüßen Julian Panke \ Original-Nachricht Am 4. Okt. 2021, 21:19, Chris Lawrence via Outages schrieb: > > > > Mailing list is seeing a lot of duplicate messages abou

Re: [outages] www.facebook.com not resolving from Spectrum/Tampa

Sorry all, disregard. I think that PM I linked was to an older outage. *Roger Self* IT Director City of Cabot 501.843.3566 (x1013) www.CabotAR.gov On Mon, Oct 4, 2021 at 5:40 PM Roger Self wrote: > Outage appears resolved. At least for now. Post Mortem >

Re: [outages] www.facebook.com not resolving from Spectrum/Tampa

Outage appears resolved. At least for now. Post Mortem *Roger Self* On Mon, Oct 4, 2021 at 5:32 PM Jay R. Ashworth via Outages < outages@outages.org> wrote: > - Original Message - > > From: "Jay R. Ashworth via Outages" > > > So i

Re: [outages] Facebook

It is looking "ok" in Texasfor now: ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12849 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;www.facebook.com.

Re: [outages] facebook outage

> On Oct 4, 2021, at 11:21 PM, Bill Woodcock wrote: > > > >> On Oct 4, 2021, at 11:10 PM, Bill Woodcock wrote: >> >> They’re starting to pick themselves back up off the floor in the last two or >> three minutes. A few answers getting out. I imagine it’ll take a while >> before things st

Re: [outages] [EXTERNAL] Re: Facebook

It’s back up On my mobile in MI Thank You, Shawn Begarowicz (Technology/Service Manager) Complete Interactive Technologies 44265 Groesbeck Highway Clinton Township, MI 48036 586-439-2000 Ext. 120 | 586-439-2001Fax 586-913-2420 Direct Tel | 586-203-4940Direct Fax 586-883-4037 Cell shawn.begarow.

Re: [outages] facebook outage

> On Oct 4, 2021, at 11:10 PM, Bill Woodcock wrote: > > They’re starting to pick themselves back up off the floor in the last two or > three minutes. A few answers getting out. I imagine it’ll take a while > before things stabilize, though. nd we’re back: WoodyNet-2:.ssh woody$ dig ww

Re: [outages] Facebook Down

All up now. -bruce From: Outages on behalf of Hugo Blom via Outages Sent: Monday, October 4, 2021 12:27:48 PM To: Matt Hoppes Cc: outages@outages.org Subject: Re: [outages] Facebook Down DNS issues seem to be the case in Europe as well. I ran a check on most

Re: [outages] facebook outage

They’re starting to pick themselves back up off the floor in the last two or three minutes. A few answers getting out. I imagine it’ll take a while before things stabilize, though. -Bill signature.asc Description: Message signed with OpenPGP _

[outages] Facebook back up

Productivity back down Due to the outages list backlog, you will probably receive this sometime after Thanksgiving -Ryan McGinnis r...@digicana.com [http://www.bigstormpicture.com](http://bigstormpicture.com) GPG: 5C73 8727 EE58 786A 777C 4F1D B5AA 3FA3 486E D7AD_

Re: [outages] Facebook Down

Facebook is slowly coming back online. They have fixed Facebook.com page, and it appears some the Android app features are working again. On Mon, Oct 4, 2021, 3:50 PM Scott Bragg via Outages wrote: > Here's what I'm seeing from one of my aws instances. Internal AWS dns > server is fine, but pub

[outages] Ars Reporting FB Issues as BGP FUBAR

https://arstechnica.com/information-technology/2021/10/facebook-instagram-whatsapp-and-oculus-are-down-heres-what-we-know/ And, it's even craizer than initially reported. DNS is down because of BGP screwup, so employee badges can't enter server room!! -- Jon R. KiblerBlog: Real World Cyber Securit

[outages] NOTICE: List was clogged

You're going to see a bunch of meetoos from around noon, which apparently were clogged up in the list machinery, dripping out now; please note the send time when evaluating whether we have list members who are *really* slow or not. :-) Cheers, -- jra -- Jay R. Ashworth Baylink

[outages] FACEBOOK - recovery begins after 6 hours

You(r users) may continue to see timeouts and errors for another hour or two, into Monday evening EDT, but at least the DNS (and hence, presumably the BGP) problems appear to be fixed or in-work. As usual for an outage of this magnitude (one that makes the general press), the list won't need indiv

Re: [outages] facebook outage

Can't wait to see the RFO! -Mike On Mon, Oct 4, 2021 at 2:35 PM Bill Woodcock wrote: > > > > > On Oct 4, 2021, at 11:21 PM, Bill Woodcock wrote: > > > > > > > >> On Oct 4, 2021, at 11:10 PM, Bill Woodcock wrote: > >> > >> They’re starting to pick themselves back up off the floor in the last tw

Re: [outages] Facebook Down

Peace, On Mon, Oct 4, 2021, 7:08 PM Matt Hoppes via Outages wrote: > Is anyone else seeing Facebook down? > b.ns.facebook.com seems to be starting working again. -- Töma > ___ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/

Re: [outages] WhatsApp down?

I received this error message: 5xx Server Error On whatsapp web. Em seg., 4 de out. de 2021 às 18:06, Osvaldo Filho < osvaldofilho.re...@gmail.com> escreveu: > I saw on a Brazilian site that they changed the DNS route. > > Em seg., 4 de out. de 2021 às 17:48, Rusty Dekema via Outages < > outage

Re: [outages] Facebook Down

Messages to this list are still late arriving... I haven't seen my own posts to the list return to me yet. The message that I'm replying to, sent at 9:15 -0700, is the most recent I've received. What could this list/outages.org have to do with Facebook being down? What else is going on? Meanti

Re: [outages] Facebook Down

> JUST IN - Facebook employees reportedly can't enter buildings to evaluate the Internet outage because their door access badges weren’t working (NYT) On Mon, Oct 4, 2021, 1:58 PM Robert Brockway via Outages < outages@outages.org> wrote: > Yes there seems to be a widespread outage impacting Fac

Re: [outages] Facebook Down

I think everyone has known that they’ve been down for hours now, Matt’s email came in 5 hours ago. Hopefully we can stop the messages now… On Mon, Oct 4, 2021 at 11:46 AM Matt Hoppes via Outages wrote: Is anyone else seeing Facebook down? Seems to be offline here in Pennsylvania.. sta

Re: [outages] WhatsApp down?

I saw on a Brazilian site that they changed the DNS route. Em seg., 4 de out. de 2021 às 17:48, Rusty Dekema via Outages < outages@outages.org> escreveu: > Yes, I think someone at Facebook may have made a mistake today: > > > facebook.com. > Server: dns.google > Address: 2001:4860:4860:: >

Re: [outages] Fwd: massive facebook outage presently

Peace, On Mon, Oct 4, 2021, 10:57 PM Bill Woodcock via Outages wrote: > Is it always the DNS? We’ll see. > This one is unique in that it's a DNS _and_ BGP issue combined. -- Töma > ___ Outages mailing list Outages@outages.org https://puck.nether.ne