Send Outages-discussion mailing list submissions to
outages-discussion@outages.org
To subscribe or unsubscribe via the World Wide Web, visit
https://puck.nether.net/mailman/listinfo/outages-discussion
or, via email, send a message with subject or body 'help' to
outages-discussion-requ...@outages.org
You can reach the person managing the list at
outages-discussion-ow...@outages.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Outages-discussion digest..."
Today's Topics:
1. Re: [outages] Amazon S3 down (Frank Bulk)
2. S3 Outages Postmortem (Bob Strecansky)
3. Re: S3 Outages Postmortem (Kevin Blackham)
4. Re: S3 Outages Postmortem (Bob Strecansky)
----------------------------------------------------------------------
Message: 1
Date: Wed, 1 Mar 2017 12:10:15 -0600
From: "Frank Bulk" <frnk...@iname.com>
To: <outages-discussion@outages.org>
Subject: Re: [Outages-discussion] [outages] Amazon S3 down
Message-ID: <001301d292b7$13ec2800$3bc47800$@iname.com>
Content-Type: text/plain; charset="utf-8"
Here is ThousandEye?s analysis (guess):
https://blog.thousandeyes.com/aws-s3-outage-likely-caused-by-internal-network-issue/
Frank
From: Outages [mailto:outages-boun...@outages.org] On Behalf Of Chapman, Brad
(NBCUniversal) via Outages
Sent: Wednesday, March 01, 2017 10:44 AM
To: outa...@outages.org
Subject: Re: [outages] Amazon S3 down
Well, that makes *two* post-mortems I look forward to reading about this month:
the Oscar night best picture kerfuffle, and now this Amazon outage.
Brad Chapman
Mac Systems Engineer
NBCUniversal
From: Outages [mailto:outages-boun...@outages.org] On Behalf Of Jake Hayhurst
via Outages
Sent: Tuesday, February 28, 2017 4:12 PM
To: Ryan Prihoda <rprih...@dynaconnections.com
<mailto:rprih...@dynaconnections.com> >
Cc: outa...@outages.org <mailto:outa...@outages.org>
Subject: Re: [outages] Amazon S3 down
Same here. We were more or less back up about 2 hours ago.
On 1 Mar 2017 12:10 a.m., "Ryan Prihoda via Outages" <outa...@outages.org
<mailto:outa...@outages.org> > wrote:
We're beginning to see recovery here.
Ryan Prihoda
Systems Administrator
dynaConnections
512.306.9898 ext 107 <tel:(512)%20306-9898>
1101 S. Capital of TX Hwy.
Bldg. H., Suite 130
Austin, Texas 78746
rprih...@dynaconnections.com <mailto:rprih...@dynaconnections.com>
www.dynaconnections.com <http://www.dynaconnections.com>
On 02/28/2017 11:55 AM, Carlos Alvarez via Outages wrote:
We are unable to reach it from Cox cable, Highwinds, or Cogent from the Phoenix
area. Also several of the outage testing sites say it's down.
--
Carlos Alvarez
602-368-6403 <tel:(602)%20368-6403>
_______________________________________________
Outages mailing list
outa...@outages.org <mailto:outa...@outages.org>
https://puck.nether.net/mailman/listinfo/outages
_______________________________________________
Outages mailing list
outa...@outages.org <mailto:outa...@outages.org>
https://puck.nether.net/mailman/listinfo/outages
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://puck.nether.net/pipermail/outages-discussion/attachments/20170301/34cb9c6d/attachment-0001.html>
------------------------------
Message: 2
Date: Wed, 01 Mar 2017 19:06:08 +0000
From: Bob Strecansky <b...@mailchimp.com>
To: outages-discussion@outages.org
Subject: [Outages-discussion] S3 Outages Postmortem
Message-ID:
<caegg-hxzvltwprdgf2joc_4av3emczcx6hick0he9wfvykc...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Has anyone heard anything about why S3 was down for 5 hours yesterday?
Usually Amazon doesn't post postmortems, and i'm curious as to what
happened.
Thanks,
Bob Strecansky
--
Thanks,
-B
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://puck.nether.net/pipermail/outages-discussion/attachments/20170301/371fd800/attachment-0001.html>
------------------------------
Message: 3
Date: Wed, 1 Mar 2017 12:17:10 -0700
From: Kevin Blackham <black...@gmail.com>
To: Bob Strecansky <b...@mailchimp.com>
Cc: outages-discussion@outages.org
Subject: Re: [Outages-discussion] S3 Outages Postmortem
Message-ID:
<can9pxvxozyu5rspboxslraul-stcnnbtjpt+1bkdnawgcx5...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
I have some insights, but I'm under NDA. This was big enough I expect some
public disclosure (my words).
I can tell you we observed lots of services affected, not just S3. EBS was
jacking up IO all over the place, and many machines didn't even ping. SES
was quite broken, as was autoscaling. One might conclude it was a network
problem.
On Mar 1, 2017 12:09, "Bob Strecansky" <b...@mailchimp.com> wrote:
> Has anyone heard anything about why S3 was down for 5 hours yesterday?
> Usually Amazon doesn't post postmortems, and i'm curious as to what
> happened.
>
> Thanks,
>
> Bob Strecansky
> --
> Thanks,
>
> -B
>
> _______________________________________________
> Outages-discussion mailing list
> Outages-discussion@outages.org
> https://puck.nether.net/mailman/listinfo/outages-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://puck.nether.net/pipermail/outages-discussion/attachments/20170301/5bfc2f87/attachment-0001.html>
------------------------------
Message: 4
Date: Wed, 01 Mar 2017 19:20:56 +0000
From: Bob Strecansky <b...@mailchimp.com>
To: Kevin Blackham <black...@gmail.com>
Cc: outages-discussion@outages.org
Subject: Re: [Outages-discussion] S3 Outages Postmortem
Message-ID:
<caegg-hwvgrhs403yahm04_w7pcoevn_vxtlb6rkabp-wf_t...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
I was thinking it had to most likely be a networking problem; considering
that it was so abrupt to come off and come on.
On Wed, Mar 1, 2017 at 2:17 PM Kevin Blackham <black...@gmail.com> wrote:
> I have some insights, but I'm under NDA. This was big enough I expect some
> public disclosure (my words).
>
> I can tell you we observed lots of services affected, not just S3. EBS was
> jacking up IO all over the place, and many machines didn't even ping. SES
> was quite broken, as was autoscaling. One might conclude it was a network
> problem.
>
> On Mar 1, 2017 12:09, "Bob Strecansky" <b...@mailchimp.com> wrote:
>
> Has anyone heard anything about why S3 was down for 5 hours yesterday?
> Usually Amazon doesn't post postmortems, and i'm curious as to what
> happened.
>
> Thanks,
>
> Bob Strecansky
> --
> Thanks,
>
> -B
>
> _______________________________________________
> Outages-discussion mailing list
> Outages-discussion@outages.org
> https://puck.nether.net/mailman/listinfo/outages-discussion
>
> --
Thanks,
-B
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://puck.nether.net/pipermail/outages-discussion/attachments/20170301/75aad97e/attachment.html>
------------------------------
Subject: Digest Footer
_______________________________________________
Outages-discussion mailing list
Outages-discussion@outages.org
https://puck.nether.net/mailman/listinfo/outages-discussion
------------------------------
End of Outages-discussion Digest, Vol 93, Issue 1
*************************************************
