I think the more plausible explanation is that Zuckerberg was in a bar
with some guys he was trying to impress saying something like 'I do SO
run Facebook' and then whips out his phone and instead of deleting their
profile, he presses the 'delete all' button by accident.
I've been at some small businesses where they can't change an Ethernet
port speed without a 3 person peer review of the config update, I have
trouble believing that one guy mistyped something.
On 10/5/2021 12:17 PM, Steve Jones wrote:
ill bet whatever their process was, its not what it is and delayed
commits/fallbacks are instituted forthwith. Not by the guy who
committed this set of changes though, hes at the bottom of a river now
On Tue, Oct 5, 2021 at 12:13 PM Cassidy B. Larson <c...@infowest.com
<mailto:c...@infowest.com>> wrote:
I wonder if they did a "commit confirmed 30”? If not, would that
have avoided the whole thing?
On Oct 5, 2021, at 11:08 AM, Carl Peterson
<cpeter...@portnetworks.com <mailto:cpeter...@portnetworks.com>>
wrote:
"an engineer put in a config change on a BGP router that
propagated to the other BGP routers and cut off access"
Sounds like something I would do which is why I generally insist
on having a remote tech physically in the building with a console
cable if I don't have OOB console access when changing BGP on my
edge.
On Tue, Oct 5, 2021 at 11:34 AM Robert <i...@avantwireless.com
<mailto:i...@avantwireless.com>> wrote:
FB has posted that an engineer put in a config change on a
BGP router that propagated to the other BGP routers and cut
off access. Sounds like it took a while to track down the
config change. I do know someone who is intimate with that
group but he's not talking... Drinking, but not talking...
On 10/5/21 9:17 AM, Bill Prince wrote:
I see it differently. In the beginning cloudflare thought
something was wrong with their systems. It doesn't explain
what happened inside facebook, probably because only
facebook knows what went wrong. That's not cloudflare's
problem. We may never know what happened at facebook
probably because it's very embarrassing to facebook.
bp
<part15sbs{at}gmail{dot}com>
On 10/5/2021 9:01 AM, Robert wrote:
I really don't like that cloudflare posted this, as it
doesn't do anything to tell why. It's not a root cause,
it's an effect and cloudflare looks like they are just
trying to say how smart they are.
On 10/5/21 8:40 AM, Bill Prince wrote:
https://blog.cloudflare.com/october-2021-facebook-outage/
bp
<part15sbs{at}gmail{dot}com>
On 10/5/2021 7:39 AM, Chuck McCown via AF wrote:
Sign him up for a SNAP card.
Sent from my iPhone
On Oct 5, 2021, at 8:25 AM, Bill Prince
<part15...@gmail.com> <mailto:part15...@gmail.com> wrote:
Heard the Zuch got around $6 or $7 billion trimmed from
his net worth yesterday. Now he's only worth around $121
billion.
bp
<part15sbs{at}gmail{dot}com>
On 10/5/2021 6:52 AM, dmmoff...@gmail.com
<mailto:dmmoff...@gmail.com> wrote:
I am very curious what it was that took 6 hours to
figure out or undo.
…..but I wouldn’t judge too harshly. When any
contraption gets big and complex it’s easy to overlook
one seemingly small thing.
Remember this XKCD?
<image001.png>
*From:* AF <af-boun...@af.afmug.com>
<mailto:af-boun...@af.afmug.com> *On Behalf Of *Steve Jones
*Sent:* Monday, October 04, 2021 6:51 PM
*To:* AnimalFarm Microwave Users Group
<af@af.afmug.com> <mailto:af@af.afmug.com>
*Subject:* Re: [AFMUG] facebook outage
so it turns out FB murdered the announcements for the
DNS server prefixes, including the registrar.
https://blog.cloudflare.com/october-2021-facebook-outage/?fbclid=IwAR1NiZ1N3M-tYQzvTVum8bV5OGpi8nLZkb7lqANzioJ7lkG6mjsqTHvhHMQ
This really seems like an elementary design flaw. Even
a moron like me knows not to poke the DNS subnets all
at once, and we probably get way less than 1k daily
lookups across all the domains we host DNS for. Ill bet
nobody involved in this makes less than 80k a year.
facebook lost 4% stock value today attributed to this
on top of the almost percent attributed to the
"whistleblower". I get nervous when I lose the company
a number 10 wrench.
On Mon, Oct 4, 2021 at 4:51 PM CBB - Jay Fuller
<par...@cyberbroadband.net
<mailto:par...@cyberbroadband.net>> wrote:
it is alive here again
----- Original Message -----
*From:*CBB - Jay Fuller
<mailto:par...@cyberbroadband.net>
*To:*AnimalFarm Microwave Users Group
<mailto:af@af.afmug.com>
*Sent:*Monday, October 4, 2021 4:44 PM
*Subject:*Re: [AFMUG] facebook outage
yup. resolving here. but nothing coming up yet.
----- Original Message -----
*From:*Mike Hammett <mailto:af...@ics-il.net>
*To:*AnimalFarm Microwave Users Group
<mailto:af@af.afmug.com>
*Sent:*Monday, October 4, 2021 4:36 PM
*Subject:*Re: [AFMUG] facebook outage
They're back... ish.
-----
Mike Hammett
Intelligent Computing Solutions
<http://www.ics-il.com/>
<https://www.facebook.com/ICSIL>
<~WRD0002.jpg> <https://www.facebook.com/ICSIL>
<~WRD0002.jpg>
<https://plus.google.com/+IntelligentComputingSolutionsDeKalb>
<~WRD0002.jpg>
<https://www.linkedin.com/company/intelligent-computing-solutions>
<~WRD0002.jpg>
<https://twitter.com/ICSIL>
Midwest Internet Exchange
<http://www.midwest-ix.com/>
<~WRD0002.jpg>
<https://www.facebook.com/mdwestix>
<~WRD0002.jpg>
<https://www.linkedin.com/company/midwest-internet-exchange>
<~WRD0002.jpg>
<https://twitter.com/mdwestix>
The Brothers WISP
<http://www.thebrotherswisp.com/>
<~WRD0002.jpg>
<https://www.facebook.com/thebrotherswisp>
<~WRD0002.jpg>
<https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg>
------------------------------------------------------------------------
*From: *"CBB - Jay Fuller"
<par...@cyberbroadband.net
<mailto:par...@cyberbroadband.net>>
*To: *af@af.afmug.com <mailto:af@af.afmug.com>
*Sent: *Monday, October 4, 2021 1:13:01 PM
*Subject: *[AFMUG] facebook outage
anyone else feeling alone and isolated
without facebook? I need my safe space
(no, not myspace) back.
babaahahah.
--
AF mailing list
AF@af.afmug.com <mailto:AF@af.afmug.com>
http://af.afmug.com/mailman/listinfo/af_af.afmug.com
------------------------------------------------------------------------
--
AF mailing list
AF@af.afmug.com <mailto:AF@af.afmug.com>
http://af.afmug.com/mailman/listinfo/af_af.afmug.com
------------------------------------------------------------------------
--
AF mailing list
AF@af.afmug.com <mailto:AF@af.afmug.com>
http://af.afmug.com/mailman/listinfo/af_af.afmug.com
--
AF mailing list
AF@af.afmug.com <mailto:AF@af.afmug.com>
http://af.afmug.com/mailman/listinfo/af_af.afmug.com
--
AF mailing list
AF@af.afmug.com <mailto:AF@af.afmug.com>
http://af.afmug.com/mailman/listinfo/af_af.afmug.com
--
AF mailing list
AF@af.afmug.com <mailto:AF@af.afmug.com>
http://af.afmug.com/mailman/listinfo/af_af.afmug.com
--
Carl Peterson
*PORT NETWORKS*
401 E Pratt St, Ste 2553
Baltimore, MD 21202
(410) 637-3707
--
AF mailing list
AF@af.afmug.com <mailto:AF@af.afmug.com>
http://af.afmug.com/mailman/listinfo/af_af.afmug.com
--
AF mailing list
AF@af.afmug.com <mailto:AF@af.afmug.com>
http://af.afmug.com/mailman/listinfo/af_af.afmug.com
--
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com
