Send Outages-discussion mailing list submissions to
outages-discussion@outages.org
To subscribe or unsubscribe via the World Wide Web, visit
https://puck.nether.net/mailman/listinfo/outages-discussion
or, via email, send a message with subject or body 'help' to
outages-discussion-requ...@outages.org
You can reach the person managing the list at
outages-discussion-ow...@outages.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Outages-discussion digest..."
Today's Topics:
1. Re: [outages] Evocative EWR1 (formerly INAP NYJ004) (Phil K.)
2. Re: [outages] Peering out of Spectrum/Charter issues?
(George Metz)
----------------------------------------------------------------------
Message: 1
Date: Tue, 11 Jul 2023 21:54:50 -0400
From: "Phil K." <clipper...@gmail.com>
Cc: outages-discussion <outages-discussion@outages.org>
Subject: Re: [Outages-discussion] [outages] Evocative EWR1 (formerly
INAP NYJ004)
Message-ID:
<CAAxiKOXT1=fyatkbijviyoon0bth00thcenyhu9wq-2dixh...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
The outage is also very dependant on what tripped power in the first place.
We've been on the receiving end of multiple outages owing to a wonky &
poorly documented Emergency Power Off circuit in a datacenter. If the fire
suppression system engages, power to some / most / all systems (depending
on configuration) gets cut to protect firefighters. In our case, there was
*supposed* to be a bypass for the EPO, e.g. "turn this key to prevent the
EPO from engaging when you do your annual fire alarm testing & maintenance."
Well, that testing procedure hadn't been fully followed for a number of
years. Someone finally got a wild hair and decided to do it by the book.
Turn the key...engage the fire alarm test...(whump) dark datacenter...turns
out that system didn't *quite* follow the schematics the old vendor
provided. But at least the EPO functioned exactly as it was supposed to.
Good times...good times...
New Jersey also lost a pair of firefighters this week while responding to a
ship board fire. They're going to be a might touchy about firefighter
safety issues for a while.
~Phil
On Tue, Jul 11, 2023, 8:24 PM Matthew Petach via Outages-discussion <
outages-discussion@outages.org> wrote:
>
> Many, many years ago when I was in a similar position, the fire marshal
> told me in no uncertain terms that if I fired up our genset before he gave
> the all-clear, he had the authority revoke our occupancy permit for the
> building. As fire marshals are often sworn members of law enforcement,
> authorized to carry a weapon and enforce the law, I felt it would be
> prudent to follow their directives and leave the building dark until they
> gave the all-clear.
>
> I don't think the folks there want to jeopardize their business by jumping
> the gun and disobeying the fire marshal. ^_^;
>
>
> On Tue, Jul 11, 2023 at 5:09?PM Charles Sprickman via Outages-discussion <
> outages-discussion@outages.org> wrote:
>
>> This is New Jersey, sounds like some palms have not been properly
>> greased. :)
>>
>> "Nice datacenter you got here, shame if it didn't have some
>> electricity..."
>>
>> Curious what would happen if they just opted to power from the gensets.
>>
>> Charles
>>
>> > On Jul 11, 2023, at 7:57 PM, Christopher J. Pilkington via Outages <
>> outa...@outages.org> wrote:
>> >
>> > On Tue, Jul 11, 2023, at 08:52, Alex via Outages wrote:
>> >> Looks like EWR1 will be down at least until the afternoon.
>> >
>> > Latest update, 9am meet with fire marshal. Best case, customer loads
>> energized 4-5 hours after.
>> >
>> > -cjp
>> > _______________________________________________
>> > Outages mailing list
>> > outa...@outages.org
>> > https://puck.nether.net/mailman/listinfo/outages
>>
>> _______________________________________________
>> Outages-discussion mailing list
>> Outages-discussion@outages.org
>> https://puck.nether.net/mailman/listinfo/outages-discussion
>>
> _______________________________________________
> Outages-discussion mailing list
> Outages-discussion@outages.org
> https://puck.nether.net/mailman/listinfo/outages-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://puck.nether.net/pipermail/outages-discussion/attachments/20230711/9c6aaf18/attachment-0001.htm>
------------------------------
Message: 2
Date: Wed, 12 Jul 2023 14:46:01 -0400
From: George Metz <george.m...@gmail.com>
To: "Warren, David" <dwar...@winstead.com>, outages-discussion
<outages-discussion@outages.org>
Cc: "outa...@outages.org" <outa...@outages.org>
Subject: Re: [Outages-discussion] [outages] Peering out of
Spectrum/Charter issues?
Message-ID:
<CANjVB-ihqcR3m=P4etKNXk=u0w=2ehk+ggqpyy7ksqb1jcw...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
We also were seeing this in Maine - a few minutes ago it looks like
Spectrum stopped handing off to HE and all the latency and packet loss we
were seeing dropped off.
On Wed, Jul 12, 2023 at 2:33?PM Warren, David via Outages <
outa...@outages.org> wrote:
>
>
> We have seen similar. We have seen increased spikes in latency and drops
> as well to two of our datacenters in Texas. The common theme for our users
> with high latency and drops is they are using Spectrum home internet.
> Everyone not using Spectrum is not having any issues. We are mainly located
> in Texas.
>
>
>
>
>
> *From:* Outages <outages-boun...@outages.org> *On Behalf Of *James Sharp
> via Outages
> *Sent:* Wednesday, July 12, 2023 12:50 PM
> *To:* outa...@outages.org
> *Subject:* [outages] Peering out of Spectrum/Charter issues?
>
>
>
>
>
> We're seeing massive latency increases (normal 50-60ms, now up to
> 4000-6000ms) between our customers and our datacenter in LA, mostly from
> customers and end users on Spectrum. Downdetector is showing a spike in
> issues. Anyone else seeing something similar?
>
> --
> James Sharp
>
> Senior Voice Engineer
>
> Engineering | Convoso, Inc
>
> [image: Image removed by sender.]
>
> 818-337-6365 <%20818-337-6365>
>
> [image: Image removed by sender.]
>
> jsh...@convoso.com
>
> [image: Image removed by sender.]
>
> www.convoso.com
>
> [image: Image removed by sender.]
>
> 22837 Ventura Blvd. Suite 300 Woodland Hills, CA 91364
>
> [image: Image removed by sender.]
>
>
>
> ------------------------------
>
> Information contained in this transmission is attorney privileged and
> confidential. It is intended for the use of the individual or entity named
> above. If the reader of this message is not the intended recipient, you are
> hereby notified that any dissemination, distribution or copy of this
> communication is strictly prohibited. If you have received this
> communication in error, please immediately notify us by telephone.
> _______________________________________________
> Outages mailing list
> outa...@outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://puck.nether.net/pipermail/outages-discussion/attachments/20230712/a460d69c/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ~WRD0004.jpg
Type: image/jpeg
Size: 823 bytes
Desc: not available
URL:
<https://puck.nether.net/pipermail/outages-discussion/attachments/20230712/a460d69c/attachment.jpg>
------------------------------
Subject: Digest Footer
_______________________________________________
Outages-discussion mailing list
Outages-discussion@outages.org
https://puck.nether.net/mailman/listinfo/outages-discussion
------------------------------
End of Outages-discussion Digest, Vol 157, Issue 2
**************************************************
