Re: Rogers Outage Canada

[Snowmobile2004]

;>> Lets deal with REAL threats not PC references.
>>>>
>>>> Time to evaluate n re eval Our NTS!!
>>>>
>>>> Veni vidi vici 2022
>>>>
>>>>
>>>>
>>>>
>>>> On Sat, Jul 9, 2022 at 5:01 PM Eric Kuhnke 
>>>> wrote:
>>>>
>>>>> Can we have a discussion with the list admins about a list member
>>>>> appending a threat of violence to their outbound emails? Whether serious 
>>>>> or
>>>>> not.
>>>>>
>>>>> Does this person need directions to some local mental healthcare
>>>>> resources?
>>>>>
>>>>>
>>>>> On Sat, 9 Jul 2022 at 08:48, Keith Medcalf 
>>>>> wrote:
>>>>>
>>>>>>
>>>>>> >I can't either, but the reality right now seems to be that 911 calls
>>>>>> are
>>>>>> >failing for anyone on a Rogers cellphone.
>>>>>>
>>>>>> This is par for the course. These people chose to deal with Rogers
>>>>>> despite knowing the consequences. It is like if you bought a Rogers
>>>>>> Snowblower and it did not work. That would mean that people who bought 
>>>>>> the
>>>>>> Rogers Snowblower will not be using it to get rid of the snow that is
>>>>>> preventing them from leaving their house.
>>>>>>
>>>>>> Mutatis mutandis when Rogers is down things that are Rogers dependent
>>>>>> will not work.
>>>>>>
>>>>>> Some people are so retarded it is astonishing!
>>>>>>
>>>>>> --
>>>>>> (CAUTION) You are advised that if you attack my person or property,
>>>>>> you will be put down in accordance with the provisions of section 34 & 35
>>>>>> of the Criminal Code respectively. If you are brandishing (or in
>>>>>> possession) of a weapon then lethal force will be applied to your person 
>>>>>> in
>>>>>> accordance with the law. This means that your misadventures may end in 
>>>>>> your
>>>>>> death. Consider yourself cautioned and govern your actions appropriately.
>>>>>>
>>>>>> >-Original Message-
>>>>>> >From: NANOG  On Behalf
>>>>>> Of
>>>>>> >Eric Kuhnke
>>>>>> >Sent: Friday, 8 July, 2022 13:34
>>>>>> >To: jim deleskie 
>>>>>> >Cc: NANOG list 
>>>>>> >Subject: Re: Rogers Outage Canada
>>>>>> >
>>>>>> >
>>>>>> >I have seen anecdotal reports that the mobile network is in a half
>>>>>> broken
>>>>>> >state that phones remain registered to, so a 911 call will attempt
>>>>>> and
>>>>>> >then fail.
>>>>>> >
>>>>>> >
>>>>>> >This is unlike what would happen if you had a US/Canada cellphone
>>>>>> with
>>>>>> >battery power but no SIM card in it that would search for any
>>>>>> available
>>>>>> >network in RF range for a 911 call if needed.
>>>>>> >
>>>>>> >
>>>>>> >On Fri, 8 Jul 2022 at 12:31, jim deleskie >>>>> ><mailto:deles...@gmail.com> > wrote:
>>>>>> >
>>>>>> >
>>>>>> > i cant see BGP taking out SS7.
>>>>>> >
>>>>>> > -jim
>>>>>> >
>>>>>> > On Fri, Jul 8, 2022 at 2:45 PM Snowmobile2004
>>>>>> >mailto:greenjosh6...@gmail.com> > wrote:
>>>>>> >
>>>>>> >
>>>>>> > According to Cloudflare Radar
>>>>>> ><https://radar.cloudflare.com/asn/812?date_filter=last_24_hours> ,
>>>>>> Rogers
>>>>>> >BGP announcements spiked massively to levels 536,777% higher than
>>>>>> normal
>>>>>> >(343,601 vs 64 normally) just minutes before the outage. I would not
>>>>>> be
>>>>>> >surprised if this happened to be the culprit.
>>>>>> >
>>>>>> > Regards,
>>>>>> > Josh Green
>>>>>> >
>>>>>> > On Fri, Jul 8, 2022 at 2:19 PM Andrew Paolucci via NANOG
>>>>>> >mailto:nanog@nanog.org> > wrote:
>>>>>> >
>>>>>> >
>>>>>> > In the early hours of the morning around 2-3am my modem
>>>>>> >got hit with a configuration update that caused a DHCP release that
>>>>>> >wasn't renewed for about two hours, after rollback the connection was
>>>>>> >fine for 3 hours before this network wide outage.
>>>>>> >
>>>>>> >
>>>>>> > Maybe a failed night time update was attempted again
>>>>>> >during office hours, I've heard daytime guys are still WFH and night
>>>>>> >shift is in building.
>>>>>> >
>>>>>> >
>>>>>> > I expect we'll never get a real explanation. Rogers is
>>>>>> >notorious for withholding any type of helpful or technical
>>>>>> information.
>>>>>> >
>>>>>> >
>>>>>> > Sent from my inoperable Rogers Mobile via emergency eSIM.
>>>>>> >
>>>>>> >
>>>>>> > Regards,
>>>>>> >
>>>>>> > Andrew Paolucci
>>>>>> >  Original Message 
>>>>>> > On Jul. 8, 2022, 1:48 p.m., Jay Hennigan < j...@west.net
>>>>>> ><mailto:j...@west.net> > wrote:
>>>>>> >
>>>>>> >
>>>>>> > On 7/8/22 07:44, Robert DeVita wrote: > Does anyone
>>>>>> >have information on a widespread Rogers outage in Canada. I > have
>>>>>> >customers with multiple sites down. There's discussion on the Outages
>>>>>> >mailing list. Seems widespread, affecting all services, mobile,
>>>>>> voice,
>>>>>> >Internet. No cause or ETR posted yet. -- Jay Hennigan - j...@west.net
>>>>>> ><mailto:j...@west.net> Network Engineering - CCIE #7880 503 897-8550
>>>>>> -
>>>>>> >WB6RDV
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > --
>>>>>> >
>>>>>> > Josh Green.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>
>>>
>>> --
>>> Jon Sands
>>> MFI Labshttps://fohdeesha.com/
>>>
>>>
>
> --
> Liz ***
> 416.660.5456
>
>
> --
*Josh Green.*

Re: Rogers Outage Canada

[Andrew Paolucci via NANOG]

  wrote:
>>>>>
>>>>>> Can we have a discussion with the list admins about a list member 
>>>>>> appending a threat of violence to their outbound emails? Whether serious 
>>>>>> or not.
>>>>>>
>>>>>> Does this person need directions to some local mental healthcare 
>>>>>> resources?
>>>>>>
>>>>>> On Sat, 9 Jul 2022 at 08:48, Keith Medcalf  wrote:
>>>>>>
>>>>>>>>I can't either, but the reality right now seems to be that 911 calls are
>>>>>>>>failing for anyone on a Rogers cellphone.
>>>>>>>
>>>>>>> This is par for the course. These people chose to deal with Rogers 
>>>>>>> despite knowing the consequences. It is like if you bought a Rogers 
>>>>>>> Snowblower and it did not work. That would mean that people who bought 
>>>>>>> the Rogers Snowblower will not be using it to get rid of the snow that 
>>>>>>> is preventing them from leaving their house.
>>>>>>>
>>>>>>> Mutatis mutandis when Rogers is down things that are Rogers dependent 
>>>>>>> will not work.
>>>>>>>
>>>>>>> Some people are so retarded it is astonishing!
>>>>>>>
>>>>>>> --
>>>>>>> (CAUTION) You are advised that if you attack my person or property, you 
>>>>>>> will be put down in accordance with the provisions of section 34 & 35 
>>>>>>> of the Criminal Code respectively. If you are brandishing (or in 
>>>>>>> possession) of a weapon then lethal force will be applied to your 
>>>>>>> person in accordance with the law. This means that your misadventures 
>>>>>>> may end in your death. Consider yourself cautioned and govern your 
>>>>>>> actions appropriately.
>>>>>>>
>>>>>>>>-Original Message-
>>>>>>>>From: NANOG  On Behalf Of
>>>>>>>>Eric Kuhnke
>>>>>>>>Sent: Friday, 8 July, 2022 13:34
>>>>>>>>To: jim deleskie 
>>>>>>>>Cc: NANOG list 
>>>>>>>>Subject: Re: Rogers Outage Canada
>>>>>>>>
>>>>>>>>
>>>>>>>>I have seen anecdotal reports that the mobile network is in a half 
>>>>>>>>broken
>>>>>>>>state that phones remain registered to, so a 911 call will attempt and
>>>>>>>>then fail.
>>>>>>>>
>>>>>>>>
>>>>>>>>This is unlike what would happen if you had a US/Canada cellphone with
>>>>>>>>battery power but no SIM card in it that would search for any available
>>>>>>>>network in RF range for a 911 call if needed.
>>>>>>>>
>>>>>>>>
>>>>>>>>On Fri, 8 Jul 2022 at 12:31, jim deleskie >>>>>>><mailto:deles...@gmail.com> > wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> i cant see BGP taking out SS7.
>>>>>>>>
>>>>>>>> -jim
>>>>>>>>
>>>>>>>> On Fri, Jul 8, 2022 at 2:45 PM Snowmobile2004
>>>>>>>>mailto:greenjosh6...@gmail.com> > wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> According to Cloudflare Radar
>>>>>>>><https://radar.cloudflare.com/asn/812?date_filter=last_24_hours> , 
>>>>>>>>Rogers
>>>>>>>>BGP announcements spiked massively to levels 536,777% higher than normal
>>>>>>>>(343,601 vs 64 normally) just minutes before the outage. I would not be
>>>>>>>>surprised if this happened to be the culprit.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Josh Green
>>>>>>>>
>>>>>>>> On Fri, Jul 8, 2022 at 2:19 PM Andrew Paolucci via NANOG
>>>>>>>>mailto:nanog@nanog.org> > wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> In the early hours of the morning around 2-3am my modem
>>>>>>>>got hit with a configuration update that caused a DHCP release that
>>>>>>>>wasn't renewed for about two hours, after rollback the connection was
>>>>>>>>fine for 3 hours before this network wide outage.
>>>>>>>>
>>>>>>>>
>>>>>>>> Maybe a failed night time update was attempted again
>>>>>>>>during office hours, I've heard daytime guys are still WFH and night
>>>>>>>>shift is in building.
>>>>>>>>
>>>>>>>>
>>>>>>>> I expect we'll never get a real explanation. Rogers is
>>>>>>>>notorious for withholding any type of helpful or technical information.
>>>>>>>>
>>>>>>>>
>>>>>>>> Sent from my inoperable Rogers Mobile via emergency eSIM.
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Andrew Paolucci
>>>>>>>>  Original Message 
>>>>>>>> On Jul. 8, 2022, 1:48 p.m., Jay Hennigan < j...@west.net
>>>>>>>><mailto:j...@west.net> > wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> On 7/8/22 07:44, Robert DeVita wrote: > Does anyone
>>>>>>>>have information on a widespread Rogers outage in Canada. I > have
>>>>>>>>customers with multiple sites down. There's discussion on the Outages
>>>>>>>>mailing list. Seems widespread, affecting all services, mobile, voice,
>>>>>>>>Internet. No cause or ETR posted yet. -- Jay Hennigan - j...@west.net
>>>>>>>><mailto:j...@west.net> Network Engineering - CCIE #7880 503 897-8550 -
>>>>>>>>WB6RDV
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> Josh Green.
>>>
>>> --
>>> Jon Sands
>>> MFI Labs
>>> https://fohdeesha.com/
>
> --
>
> Liz ***
> 416.660.5456

Re: Rogers Outage Canada

[L F]

ese people chose to deal with Rogers
>>>>> despite knowing the consequences.  It is like if you bought a Rogers
>>>>> Snowblower and it did not work.  That would mean that people who bought 
>>>>> the
>>>>> Rogers Snowblower will not be using it to get rid of the snow that is
>>>>> preventing them from leaving their house.
>>>>>
>>>>> Mutatis mutandis when Rogers is down things that are Rogers dependent
>>>>> will not work.
>>>>>
>>>>> Some people are so retarded it is astonishing!
>>>>>
>>>>> --
>>>>> (CAUTION) You are advised that if you attack my person or property,
>>>>> you will be put down in accordance with the provisions of section 34 & 35
>>>>> of the Criminal Code respectively.  If you are brandishing (or in
>>>>> possession) of a weapon then lethal force will be applied to your person 
>>>>> in
>>>>> accordance with the law.  This means that your misadventures may end in
>>>>> your death.  Consider yourself cautioned and govern your actions
>>>>> appropriately.
>>>>>
>>>>> >-Original Message-
>>>>> >From: NANOG  On Behalf
>>>>> Of
>>>>> >Eric Kuhnke
>>>>> >Sent: Friday, 8 July, 2022 13:34
>>>>> >To: jim deleskie 
>>>>> >Cc: NANOG list 
>>>>> >Subject: Re: Rogers Outage Canada
>>>>> >
>>>>> >
>>>>> >I have seen anecdotal reports that the mobile network is in a half
>>>>> broken
>>>>> >state that phones remain registered to, so a 911 call will attempt and
>>>>> >then fail.
>>>>> >
>>>>> >
>>>>> >This is unlike what would happen if you had a US/Canada cellphone with
>>>>> >battery power but no SIM card in it that would search for any
>>>>> available
>>>>> >network in RF range for a 911 call if needed.
>>>>> >
>>>>> >
>>>>> >On Fri, 8 Jul 2022 at 12:31, jim deleskie >>>> ><mailto:deles...@gmail.com> > wrote:
>>>>> >
>>>>> >
>>>>> >   i cant see BGP taking out SS7.
>>>>> >
>>>>> >   -jim
>>>>> >
>>>>> >   On Fri, Jul 8, 2022 at 2:45 PM Snowmobile2004
>>>>> >mailto:greenjosh6...@gmail.com> > wrote:
>>>>> >
>>>>> >
>>>>> >   According to Cloudflare Radar
>>>>> ><https://radar.cloudflare.com/asn/812?date_filter=last_24_hours> ,
>>>>> Rogers
>>>>> >BGP announcements spiked massively to levels 536,777% higher than
>>>>> normal
>>>>> >(343,601 vs 64 normally) just minutes before the outage. I would not
>>>>> be
>>>>> >surprised if this happened to be the culprit.
>>>>> >
>>>>> >   Regards,
>>>>> >   Josh Green
>>>>> >
>>>>> >   On Fri, Jul 8, 2022 at 2:19 PM Andrew Paolucci via
>>>>> NANOG
>>>>> >mailto:nanog@nanog.org> > wrote:
>>>>> >
>>>>> >
>>>>> >   In the early hours of the morning around 2-3am
>>>>> my modem
>>>>> >got hit with a configuration update that caused a DHCP release that
>>>>> >wasn't renewed for about two hours, after rollback the connection was
>>>>> >fine for 3 hours before this network wide outage.
>>>>> >
>>>>> >
>>>>> >   Maybe a failed night time update was attempted
>>>>> again
>>>>> >during office hours, I've heard daytime guys are still WFH and night
>>>>> >shift is in building.
>>>>> >
>>>>> >
>>>>> >   I expect we'll never get a real explanation.
>>>>> Rogers is
>>>>> >notorious for withholding any type of helpful or technical
>>>>> information.
>>>>> >
>>>>> >
>>>>> >   Sent from my inoperable Rogers Mobile via
>>>>> emergency eSIM.
>>>>> >
>>>>> >
>>>>> >   Regards,
>>>>> >
>>>>> >   Andrew Paolucci
>>>>> >    Original Message 
>>>>> >   On Jul. 8, 2022, 1:48 p.m., Jay Hennigan <
>>>>> j...@west.net
>>>>> ><mailto:j...@west.net> > wrote:
>>>>> >
>>>>> >
>>>>> >   On 7/8/22 07:44, Robert DeVita wrote:
>>>>> > Does anyone
>>>>> >have information on a widespread Rogers outage in Canada. I > have
>>>>> >customers with multiple sites down. There's discussion on the Outages
>>>>> >mailing list. Seems widespread, affecting all services, mobile, voice,
>>>>> >Internet. No cause or ETR posted yet. -- Jay Hennigan - j...@west.net
>>>>> ><mailto:j...@west.net>  Network Engineering - CCIE #7880 503 897-8550
>>>>> -
>>>>> >WB6RDV
>>>>> >
>>>>> >
>>>>> >
>>>>> >   --
>>>>> >
>>>>> >   Josh Green.
>>>>>
>>>>>
>>>>>
>>>>>
>>
>>
>> --
>> Jon Sands
>> MFI Labshttps://fohdeesha.com/
>>
>>

-- 
  Liz ***
416.660.5456

Re: Rogers Outage Canada

[L F]

nAR7N3DW6NYQSfVMw~~/AANj1QA~/RgRkruHwP4StAWh0dHBzOi8vd3d3LmNoYW5nZS5vcmcvcC9jdXRjZWxsY29zdHMtY2FuYWRpYW5zLWFyZS1iZWluZy1zY2FtbWVkLXRlbGwtdGhlLWNydGMtd2UtZGVtYW5kLWxvd2VyLWNlbGwtcGhvbmUtc2VydmljZS1jcnRjZW5nL3NpZ24_Y3NfdGs9QWp0bEtmZjYtX3hBU215WDFXSUFBWGljeXl2TnlRRUFCRjhCdkdKalZUd09fUklzdEFXSnBOV19PMVklM0QmcHQ9QVZCbGRHbDBhVzl1QU5WTDJRQUFBQUFBWXN4YzdJRG5PWWRqWkRkaU9URm1OQSUyNTNEJTI1M0Qmc291cmNlX2xvY2F0aW9uPWFhX2F1Z21lbnRlZF9zaWduX2FzayZ1dG1fY2FtcGFpZ249NjJiOWYzZDM2MzA0NGVhMWJhYWNmYmFkZDM2MjFiOTgmdXRtX2NvbnRlbnQ9aW5pdGlhbF92MF8wXzImdXRtX21lZGl1bT1lbWFpbCZ1dG1fc291cmNlPWFhX2F1Z21lbnRlZF9zaWduX2FzayZ1dG1fdGVybT1jc1cDc3BjQgpiu_VczGKYxTR0UhBkbnBjb25AZ21haWwuY29tWAQC>

*At Change.org, we believe in the voice of everyday people.* Is there
something that you want to change?
*Start a petition today
<https://click.e.change.org/f/a/pbTbHw7ifps5zEHVN4Z5sQ~~/AANj1QA~/RgRkruHwP0TzaHR0cHM6Ly93d3cuY2hhbmdlLm9yZy9zdGFydC1hLXBldGl0aW9uP2NzX3RrPUFqdGxLZmY2LV94QVNteVgxV0lBQVhpY3l5dk55UUVBQkY4QnZHSmpWVHdPX1JJc3RBV0pwTldfTzFZJTNEJnV0bV9jYW1wYWlnbj02MmI5ZjNkMzYzMDQ0ZWExYmFhY2ZiYWRkMzYyMWI5OCZ1dG1fY29udGVudD1pbml0aWFsX3YwXzBfMiZ1dG1fbWVkaXVtPWVtYWlsJnV0bV9zb3VyY2U9YWFfYXVnbWVudGVkX3NpZ25fYXNrJnV0bV90ZXJtPWNzVwNzcGNCCmK79VzMYpjFNHRSEGRucGNvbkBnbWFpbC5jb21YBAI~>*

The person (or organization) who started this petition is not affiliated
with Change.org. Change.org did not create this petition and is not
responsible for the petition content.

On Mon, Jul 11, 2022 at 5:40 AM Jon Sands  wrote:

> Given the outage was so bad it was disrupting select E911 services
> nationwide for something like 24+ hours, it's great to see such an in depth
> analysis and plan of action to prevent such things in the future. bravo
> rogers
>
> On 7/10/2022 2:55 PM, L F wrote:
>
> fyi - see BOLD.
>
> A Message from Rogers President and CEO
>
>
> Dear Valued Customer,
>
> As you know, we experienced a service outage across the Rogers, Fido,
> Chatr and Cityfone wireless networks on Friday.
>
> I am reaching out to share that our services have been restored, and our
> networks and systems are close to fully operational. Our technical teams
> are continuing to monitor for any remaining intermittent issues. I also
> want to outline an action plan we are putting in place to address what
> happened.
>
> I want to share what we know about what happened on Friday. We now believe 
> *we’ve
> narrowed the cause to a network system failure following a maintenance
> update in our core network, which caused some of our routers to
> malfunction. We disconnected the specific equipment and redirected traffic,*
> which allowed our network and services to come back online over time as we
> managed traffic volumes returning to normal levels.
>
>
>
> On Sat, Jul 9, 2022 at 9:09 PM L F  wrote:
>
>>
>> Lest we ALL MOVE ON….
>>
>>
>> Yes he said
>>
>> “RETARD” =
>>
>> The delay in processing the current status of a Situation…
>>
>>
>> Lets move on to bigger n scarier Post mortems:
>>
>>
>> Was This or Was this NOT
>>
>> A CYBERATTACK ???
>>
>> Done.
>>
>> Lets deal with REAL threats not PC references.
>>
>> Time to evaluate n re eval Our NTS!!
>>
>> Veni vidi vici 2022
>>
>>
>>
>>
>> On Sat, Jul 9, 2022 at 5:01 PM Eric Kuhnke  wrote:
>>
>>> Can we have a discussion with the list admins about a list member
>>> appending a threat of violence to their outbound emails?  Whether serious
>>> or not.
>>>
>>> Does this person need directions to some local mental healthcare
>>> resources?
>>>
>>>
>>> On Sat, 9 Jul 2022 at 08:48, Keith Medcalf  wrote:
>>>
>>>>
>>>> >I can't either, but the reality right now seems to be that 911 calls
>>>> are
>>>> >failing for anyone on a Rogers cellphone.
>>>>
>>>> This is par for the course.  These people chose to deal with Rogers
>>>> despite knowing the consequences.  It is like if you bought a Rogers
>>>> Snowblower and it did not work.  That would mean that people who bought the
>>>> Rogers Snowblower will not be using it to get rid of the snow that is
>>>> preventing them from leaving their house.
>>>>
>>>> Mutatis mutandis when Rogers is down things that are Rogers dependent
>>>> will not work.
>>>>
>>>> Some people are so retarded it is astonishing!
>>>>
>>>> --
>>>> (CAUTION) You are advised that if you attack my person or property, you
>>>> will be put down in accordance with the provisions of section 34 & 35 of
>>>> the Criminal Code resp

Re: Rogers Outage Canada

[Andrey Kostin]

It's hard to believe that a same time maintenance affecting so many 
devices in the core network could be approved. Core networks are build 
with redundancy, so that failures can't completely destroy the whole 
network. If it would be just "particular devices" they could be isolated 
much faster. In this case the outage was so huge that it affected mostly 
all network gear, maybe all devices from one vendor. And even after most 
of the services restored we still observed partial connectivity in some 
areas that means the impact was mush deeper, in some places down to 
access layer.


Kind regards,
Andrey

Shane Ronan писал(а) 2022-07-11 10:09:

What in depth analysis have you seen? Seems to me, this was a failure
in a known maintenance activity, and they simply disconnected the
devices under maintenance from the network.

Shane

Re: Rogers Outage Canada

[Niels Bakker]

* vic...@jvknet.com (Victor Kuarsingh) [Mon 11 Jul 2022, 17:17 CEST]:
This is the most they can and will say.  For liabilities reasons, 
specifics are likely not in the cards.


I doubt it. Given that emergency services were impacted you can count 
on a proper assessment being made available to a parliamentary 
inquiry. Too much broke to cover this up any further.



-- Niels.

Re: Rogers Outage Canada

[Jared Mauch]

> On Jul 11, 2022, at 11:15 AM, Victor Kuarsingh  wrote:
> 
> This is the most they can and will say.  For liabilities reasons, specifics 
> are likely not in the cards.  As most services ride over common service 
> networks, its quite possible that a network substrate failure can have a 
> number of upstream service impacts.  The point here is that the CEO is 
> directly addressing the customer base, which is needed here.
> 

Yes, unless you get a side-leak from someone in-the-know, I expect either 
direct or CRTC will be the best source.  I know a few of us are concerned about 
if we have any similar risk of impact to our environments, so we will be 
watching close for any hints.

- Jared

Re: Rogers Outage Canada

[Keith Stokes]

As a Rogers data center customer I received this on Saturday:


You have received this bulletin because you are an official contact for your 
Rogers Data Centres Server Colocation service

The nationwide network outage has now been resolved.

Rogers Data Centres is reporting all network and dependent services as 
operating normally. If you are experiencing any ongoing issues please reach out 
to the Data Centres help desk

An official Incident Report will be shared in 5 to 7 Business days.

We thank you for your patience during the restoration.


--

Keith Stokes
SalonBiz, Inc



On Jul 11, 2022, at 10:16 AM, Victor Kuarsingh 
mailto:vic...@jvknet.com>> wrote:

This is the most they can and will say.  For liabilities reasons, specifics are 
likely not in the cards.  As most services ride over common service networks, 
its quite possible that a network substrate failure can have a number of 
upstream service impacts.  The point here is that the CEO is directly 
addressing the customer base, which is needed here.

regards,

Victor K

On Mon, Jul 11, 2022 at 10:11 AM Shane Ronan 
mailto:sh...@ronan-online.com>> wrote:
What in depth analysis have you seen? Seems to me, this was a failure in a 
known maintenance activity, and they simply disconnected the devices under 
maintenance from the network.

Shane

On Mon, Jul 11, 2022 at 5:41 AM Jon Sands 
mailto:fohdee...@gmail.com>> wrote:
Given the outage was so bad it was disrupting select E911 services nationwide 
for something like 24+ hours, it's great to see such an in depth analysis and 
plan of action to prevent such things in the future. bravo rogers

On 7/10/2022 2:55 PM, L F wrote:
fyi - see BOLD.

A Message from Rogers President and CEO



Dear Valued Customer,

As you know, we experienced a service outage across the Rogers, Fido, Chatr and 
Cityfone wireless networks on Friday.

I am reaching out to share that our services have been restored, and our 
networks and systems are close to fully operational. Our technical teams are 
continuing to monitor for any remaining intermittent issues. I also want to 
outline an action plan we are putting in place to address what happened.

I want to share what we know about what happened on Friday. We now believe 
we’ve narrowed the cause to a network system failure following a maintenance 
update in our core network, which caused some of our routers to malfunction. We 
disconnected the specific equipment and redirected traffic, which allowed our 
network and services to come back online over time as we managed traffic 
volumes returning to normal levels.



On Sat, Jul 9, 2022 at 9:09 PM L F 
mailto:liz.faze...@gmail.com>> wrote:

Lest we ALL MOVE ON….


Yes he said

“RETARD” =

The delay in processing the current status of a Situation…


Lets move on to bigger n scarier Post mortems:


Was This or Was this NOT

A CYBERATTACK ???

Done.

Lets deal with REAL threats not PC references.

Time to evaluate n re eval Our NTS!!

Veni vidi vici 2022




On Sat, Jul 9, 2022 at 5:01 PM Eric Kuhnke 
mailto:eric.kuh...@gmail.com>> wrote:
Can we have a discussion with the list admins about a list member appending a 
threat of violence to their outbound emails?  Whether serious or not.

Does this person need directions to some local mental healthcare resources?


On Sat, 9 Jul 2022 at 08:48, Keith Medcalf 
mailto:kmedc...@dessus.com>> wrote:

>I can't either, but the reality right now seems to be that 911 calls are
>failing for anyone on a Rogers cellphone.

This is par for the course.  These people chose to deal with Rogers despite 
knowing the consequences.  It is like if you bought a Rogers Snowblower and it 
did not work.  That would mean that people who bought the Rogers Snowblower 
will not be using it to get rid of the snow that is preventing them from 
leaving their house.

Mutatis mutandis when Rogers is down things that are Rogers dependent will not 
work.

Some people are so retarded it is astonishing!

--
(CAUTION) You are advised that if you attack my person or property, you will be 
put down in accordance with the provisions of section 34 & 35 of the Criminal 
Code respectively.  If you are brandishing (or in possession) of a weapon then 
lethal force will be applied to your person in accordance with the law.  This 
means that your misadventures may end in your death.  Consider yourself 
cautioned and govern your actions appropriately.

>-Original Message-
>From: NANOG 
>mailto:dessus@nanog.org>> On 
>Behalf Of
>Eric Kuhnke
>Sent: Friday, 8 July, 2022 13:34
>To: jim deleskie mailto:deles...@gmail.com>>
>Cc: NANOG list mailto:nanog@nanog.org>>
>Subject: Re: Rogers Outage Canada
>
>
>I have seen anecdotal reports that the mobile network is in a half broken
>state that phones remain registered to, so a 911 call will attempt and
>then fail.
>
>
>This is unlike what would happen if you had a US/

Re: Rogers Outage Canada

[Victor Kuarsingh]

This is the most they can and will say.  For liabilities reasons, specifics
are likely not in the cards.  As most services ride over common service
networks, its quite possible that a network substrate failure can have a
number of upstream service impacts.  The point here is that the CEO is
directly addressing the customer base, which is needed here.

regards,

Victor K

On Mon, Jul 11, 2022 at 10:11 AM Shane Ronan  wrote:

> What in depth analysis have you seen? Seems to me, this was a failure in a
> known maintenance activity, and they simply disconnected the devices under
> maintenance from the network.
>
> Shane
>
> On Mon, Jul 11, 2022 at 5:41 AM Jon Sands  wrote:
>
>> Given the outage was so bad it was disrupting select E911 services
>> nationwide for something like 24+ hours, it's great to see such an in depth
>> analysis and plan of action to prevent such things in the future. bravo
>> rogers
>>
>> On 7/10/2022 2:55 PM, L F wrote:
>>
>> fyi - see BOLD.
>>
>> A Message from Rogers President and CEO
>>
>>
>> Dear Valued Customer,
>>
>> As you know, we experienced a service outage across the Rogers, Fido,
>> Chatr and Cityfone wireless networks on Friday.
>>
>> I am reaching out to share that our services have been restored, and our
>> networks and systems are close to fully operational. Our technical teams
>> are continuing to monitor for any remaining intermittent issues. I also
>> want to outline an action plan we are putting in place to address what
>> happened.
>>
>> I want to share what we know about what happened on Friday. We now
>> believe *we’ve narrowed the cause to a network system failure following
>> a maintenance update in our core network, which caused some of our routers
>> to malfunction. We disconnected the specific equipment and redirected
>> traffic,* which allowed our network and services to come back online
>> over time as we managed traffic volumes returning to normal levels.
>>
>>
>>
>> On Sat, Jul 9, 2022 at 9:09 PM L F  wrote:
>>
>>>
>>> Lest we ALL MOVE ON….
>>>
>>>
>>> Yes he said
>>>
>>> “RETARD” =
>>>
>>> The delay in processing the current status of a Situation…
>>>
>>>
>>> Lets move on to bigger n scarier Post mortems:
>>>
>>>
>>> Was This or Was this NOT
>>>
>>> A CYBERATTACK ???
>>>
>>> Done.
>>>
>>> Lets deal with REAL threats not PC references.
>>>
>>> Time to evaluate n re eval Our NTS!!
>>>
>>> Veni vidi vici 2022
>>>
>>>
>>>
>>>
>>> On Sat, Jul 9, 2022 at 5:01 PM Eric Kuhnke 
>>> wrote:
>>>
>>>> Can we have a discussion with the list admins about a list member
>>>> appending a threat of violence to their outbound emails?  Whether serious
>>>> or not.
>>>>
>>>> Does this person need directions to some local mental healthcare
>>>> resources?
>>>>
>>>>
>>>> On Sat, 9 Jul 2022 at 08:48, Keith Medcalf  wrote:
>>>>
>>>>>
>>>>> >I can't either, but the reality right now seems to be that 911 calls
>>>>> are
>>>>> >failing for anyone on a Rogers cellphone.
>>>>>
>>>>> This is par for the course.  These people chose to deal with Rogers
>>>>> despite knowing the consequences.  It is like if you bought a Rogers
>>>>> Snowblower and it did not work.  That would mean that people who bought 
>>>>> the
>>>>> Rogers Snowblower will not be using it to get rid of the snow that is
>>>>> preventing them from leaving their house.
>>>>>
>>>>> Mutatis mutandis when Rogers is down things that are Rogers dependent
>>>>> will not work.
>>>>>
>>>>> Some people are so retarded it is astonishing!
>>>>>
>>>>> --
>>>>> (CAUTION) You are advised that if you attack my person or property,
>>>>> you will be put down in accordance with the provisions of section 34 & 35
>>>>> of the Criminal Code respectively.  If you are brandishing (or in
>>>>> possession) of a weapon then lethal force will be applied to your person 
>>>>> in
>>>>> accordance with the law.  This means that your misadventures may end in
>>>>> your death.  Consider yourself cautioned and govern your actions
>>>>&

Re: Rogers Outage Canada

[Shane Ronan]

What in depth analysis have you seen? Seems to me, this was a failure in a
known maintenance activity, and they simply disconnected the devices under
maintenance from the network.

Shane

On Mon, Jul 11, 2022 at 5:41 AM Jon Sands  wrote:

> Given the outage was so bad it was disrupting select E911 services
> nationwide for something like 24+ hours, it's great to see such an in depth
> analysis and plan of action to prevent such things in the future. bravo
> rogers
>
> On 7/10/2022 2:55 PM, L F wrote:
>
> fyi - see BOLD.
>
> A Message from Rogers President and CEO
>
>
> Dear Valued Customer,
>
> As you know, we experienced a service outage across the Rogers, Fido,
> Chatr and Cityfone wireless networks on Friday.
>
> I am reaching out to share that our services have been restored, and our
> networks and systems are close to fully operational. Our technical teams
> are continuing to monitor for any remaining intermittent issues. I also
> want to outline an action plan we are putting in place to address what
> happened.
>
> I want to share what we know about what happened on Friday. We now believe 
> *we’ve
> narrowed the cause to a network system failure following a maintenance
> update in our core network, which caused some of our routers to
> malfunction. We disconnected the specific equipment and redirected traffic,*
> which allowed our network and services to come back online over time as we
> managed traffic volumes returning to normal levels.
>
>
>
> On Sat, Jul 9, 2022 at 9:09 PM L F  wrote:
>
>>
>> Lest we ALL MOVE ON….
>>
>>
>> Yes he said
>>
>> “RETARD” =
>>
>> The delay in processing the current status of a Situation…
>>
>>
>> Lets move on to bigger n scarier Post mortems:
>>
>>
>> Was This or Was this NOT
>>
>> A CYBERATTACK ???
>>
>> Done.
>>
>> Lets deal with REAL threats not PC references.
>>
>> Time to evaluate n re eval Our NTS!!
>>
>> Veni vidi vici 2022
>>
>>
>>
>>
>> On Sat, Jul 9, 2022 at 5:01 PM Eric Kuhnke  wrote:
>>
>>> Can we have a discussion with the list admins about a list member
>>> appending a threat of violence to their outbound emails?  Whether serious
>>> or not.
>>>
>>> Does this person need directions to some local mental healthcare
>>> resources?
>>>
>>>
>>> On Sat, 9 Jul 2022 at 08:48, Keith Medcalf  wrote:
>>>
>>>>
>>>> >I can't either, but the reality right now seems to be that 911 calls
>>>> are
>>>> >failing for anyone on a Rogers cellphone.
>>>>
>>>> This is par for the course.  These people chose to deal with Rogers
>>>> despite knowing the consequences.  It is like if you bought a Rogers
>>>> Snowblower and it did not work.  That would mean that people who bought the
>>>> Rogers Snowblower will not be using it to get rid of the snow that is
>>>> preventing them from leaving their house.
>>>>
>>>> Mutatis mutandis when Rogers is down things that are Rogers dependent
>>>> will not work.
>>>>
>>>> Some people are so retarded it is astonishing!
>>>>
>>>> --
>>>> (CAUTION) You are advised that if you attack my person or property, you
>>>> will be put down in accordance with the provisions of section 34 & 35 of
>>>> the Criminal Code respectively.  If you are brandishing (or in possession)
>>>> of a weapon then lethal force will be applied to your person in accordance
>>>> with the law.  This means that your misadventures may end in your death.
>>>> Consider yourself cautioned and govern your actions appropriately.
>>>>
>>>> >-Original Message-
>>>> >From: NANOG  On Behalf Of
>>>> >Eric Kuhnke
>>>> >Sent: Friday, 8 July, 2022 13:34
>>>> >To: jim deleskie 
>>>> >Cc: NANOG list 
>>>> >Subject: Re: Rogers Outage Canada
>>>> >
>>>> >
>>>> >I have seen anecdotal reports that the mobile network is in a half
>>>> broken
>>>> >state that phones remain registered to, so a 911 call will attempt and
>>>> >then fail.
>>>> >
>>>> >
>>>> >This is unlike what would happen if you had a US/Canada cellphone with
>>>> >battery power but no SIM card in it that would search for any available
>>>> >network in RF range for a 911 call if needed.
>>

Re: Rogers Outage Canada

[Alain Hebert]

    Seeing the little of that thread.

    I never understood why someone thinks their experience with Level 1 
thru 3 staff reflect the actual situation.


    That was clearly an issue with network engineering and their 
processes.  Because a 14h downtime for, as Rogers says, "a specific 
equipment" is too big of a spin.


-
Alain hebertaheb...@pubnix.net
PubNIX Inc.

50 boul. St-Charles
P.O. Box 26770 Beaconsfield, Quebec H9W 6G7
Tel: 514-990-5911http://www.pubnix.net Fax: 514-990-9443

On 7/9/22 21:18, Victor Kuarsingh wrote:
We should move on and focus on constructive comments vs. kicking an 
operator when they are down.


That said, demeaning comments, no matter what they are or whom they 
are targeted at, is not healthy and is not helpful.

regards,

Victor K

On Sat, Jul 9, 2022 at 9:11 PM L F  wrote:


Lest we ALL MOVE ON….


Yes he said

“RETARD” =

The delay in processing the current status of a Situation…


Lets move on to bigger n scarier Post mortems:


Was This or Was this NOT

A CYBERATTACK ???

Done.

Lets deal with REAL threats not PC references.

Time to evaluate n re eval Our NTS!!

Veni vidi vici 2022




On Sat, Jul 9, 2022 at 5:01 PM Eric Kuhnke 
wrote:

Can we have a discussion with the list admins about a list
member appending a threat of violence to their outbound
emails?  Whether serious or not.

Does this person need directions to some local mental
healthcare resources?


On Sat, 9 Jul 2022 at 08:48, Keith Medcalf
 wrote:


>I can't either, but the reality right now seems to be
that 911 calls are
>failing for anyone on a Rogers cellphone.

This is par for the course.  These people chose to deal
with Rogers despite knowing the consequences.  It is like
if you bought a Rogers Snowblower and it did not work. 
That would mean that people who bought the Rogers
Snowblower will not be using it to get rid of the snow
that is preventing them from leaving their house.

Mutatis mutandis when Rogers is down things that are
Rogers dependent will not work.

Some people are so retarded it is astonishing!

-- 
(CAUTION) You are advised that if you attack my person or

property, you will be put down in accordance with the
provisions of section 34 & 35 of the Criminal Code
respectively.  If you are brandishing (or in possession)
of a weapon then lethal force will be applied to your
person in accordance with the law.  This means that your
misadventures may end in your death. Consider yourself
cautioned and govern your actions appropriately.

>-Original Message-
>From: NANOG 
On Behalf Of
>Eric Kuhnke
>Sent: Friday, 8 July, 2022 13:34
>To: jim deleskie 
>Cc: NANOG list 
    >Subject: Re: Rogers Outage Canada
>
>
>I have seen anecdotal reports that the mobile network is
in a half broken
>state that phones remain registered to, so a 911 call
will attempt and
>then fail.
>
>
>This is unlike what would happen if you had a US/Canada
cellphone with
>battery power but no SIM card in it that would search for
any available
>network in RF range for a 911 call if needed.
>
>
>On Fri, 8 Jul 2022 at 12:31, jim deleskie <mailto:deles...@gmail.com> > wrote:
>
>
>       i cant see BGP taking out SS7.
>
>       -jim
>
>       On Fri, Jul 8, 2022 at 2:45 PM Snowmobile2004
>mailto:greenjosh6...@gmail.com>
> wrote:
>
>
>               According to Cloudflare Radar
><https://radar.cloudflare.com/asn/812?date_filter=last_24_hours>
, Rogers
>BGP announcements spiked massively to levels 536,777%
higher than normal
>(343,601 vs 64 normally) just minutes before the outage.
I would not be
>surprised if this happened to be the culprit.
>
>               Regards,
>               Josh Green
>
>               On Fri, Jul 8, 2022 at 2:19 PM Andrew
Paolucci via NANOG
>mailto:nanog@nanog.org> > wrote:
>
>
>                       In the ea

Re: Rogers Outage Canada

[Jon Sands]

Given the outage was so bad it was disrupting select E911 services 
nationwide for something like 24+ hours, it's great to see such an in 
depth analysis and plan of action to prevent such things in the future. 
bravo rogers


On 7/10/2022 2:55 PM, L F wrote:

fyi - see BOLD.

A Message from Rogers President and CEO



Dear Valued Customer,

As you know, we experienced a service outage across the Rogers, Fido, 
Chatr and Cityfone wireless networks on Friday.


I am reaching out to share that our services have been restored, and 
our networks and systems are close to fully operational. Our technical 
teams are continuing to monitor for any remaining intermittent issues. 
I also want to outline an action plan we are putting in place to 
address what happened.


I want to share what we know about what happened on Friday. We now 
believe *we’ve narrowed the cause to a network system failure 
following a maintenance update in our core network, which caused some 
of our routers to malfunction. We disconnected the specific equipment 
and redirected traffic,*which allowed our network and services to come 
back online over time as we managed traffic volumes returning to 
normal levels.





On Sat, Jul 9, 2022 at 9:09 PM L F  wrote:


Lest we ALL MOVE ON….


Yes he said

“RETARD” =

The delay in processing the current status of a Situation…


Lets move on to bigger n scarier Post mortems:


Was This or Was this NOT

A CYBERATTACK ???

Done.

Lets deal with REAL threats not PC references.

Time to evaluate n re eval Our NTS!!

Veni vidi vici 2022




On Sat, Jul 9, 2022 at 5:01 PM Eric Kuhnke 
wrote:

Can we have a discussion with the list admins about a list
member appending a threat of violence to their outbound
emails?  Whether serious or not.

Does this person need directions to some local mental
healthcare resources?


On Sat, 9 Jul 2022 at 08:48, Keith Medcalf
 wrote:


>I can't either, but the reality right now seems to be
that 911 calls are
>failing for anyone on a Rogers cellphone.

This is par for the course.  These people chose to deal
with Rogers despite knowing the consequences.  It is like
if you bought a Rogers Snowblower and it did not work. 
That would mean that people who bought the Rogers
Snowblower will not be using it to get rid of the snow
that is preventing them from leaving their house.

Mutatis mutandis when Rogers is down things that are
Rogers dependent will not work.

Some people are so retarded it is astonishing!

-- 
(CAUTION) You are advised that if you attack my person or

property, you will be put down in accordance with the
provisions of section 34 & 35 of the Criminal Code
respectively.  If you are brandishing (or in possession)
of a weapon then lethal force will be applied to your
person in accordance with the law.  This means that your
misadventures may end in your death. Consider yourself
cautioned and govern your actions appropriately.

>-Original Message-
>From: NANOG 
On Behalf Of
>Eric Kuhnke
>Sent: Friday, 8 July, 2022 13:34
>To: jim deleskie 
>Cc: NANOG list 
    >Subject: Re: Rogers Outage Canada
>
>
>I have seen anecdotal reports that the mobile network is
in a half broken
>state that phones remain registered to, so a 911 call
will attempt and
>then fail.
>
>
>This is unlike what would happen if you had a US/Canada
cellphone with
>battery power but no SIM card in it that would search for
any available
>network in RF range for a 911 call if needed.
>
>
>On Fri, 8 Jul 2022 at 12:31, jim deleskie <mailto:deles...@gmail.com> > wrote:
>
>
>       i cant see BGP taking out SS7.
>
>       -jim
>
>       On Fri, Jul 8, 2022 at 2:45 PM Snowmobile2004
>mailto:greenjosh6...@gmail.com>
> wrote:
>
>
>               According to Cloudflare Radar
><https://radar.cloudflare.com/asn/812?date_filter=last_24_hours>
, Rogers
>BGP announcements spiked massively to levels 536,777%
higher than normal
>(343,601 vs 64 normally) just minutes before the outage.
I would not be
>

Re: Rogers Outage Canada

[L F]

fyi - see BOLD.

A Message from Rogers President and CEO

Dear Valued Customer,

As you know, we experienced a service outage across the Rogers, Fido, Chatr
and Cityfone wireless networks on Friday.

I am reaching out to share that our services have been restored, and our
networks and systems are close to fully operational. Our technical teams
are continuing to monitor for any remaining intermittent issues. I also
want to outline an action plan we are putting in place to address what
happened.

I want to share what we know about what happened on Friday. We now
believe *we’ve
narrowed the cause to a network system failure following a maintenance
update in our core network, which caused some of our routers to
malfunction. We disconnected the specific equipment and redirected traffic,*
which allowed our network and services to come back online over time as we
managed traffic volumes returning to normal levels.



On Sat, Jul 9, 2022 at 9:09 PM L F  wrote:

>
> Lest we ALL MOVE ON….
>
>
> Yes he said
>
> “RETARD” =
>
> The delay in processing the current status of a Situation…
>
>
> Lets move on to bigger n scarier Post mortems:
>
>
> Was This or Was this NOT
>
> A CYBERATTACK ???
>
> Done.
>
> Lets deal with REAL threats not PC references.
>
> Time to evaluate n re eval Our NTS!!
>
> Veni vidi vici 2022
>
>
>
>
> On Sat, Jul 9, 2022 at 5:01 PM Eric Kuhnke  wrote:
>
>> Can we have a discussion with the list admins about a list member
>> appending a threat of violence to their outbound emails?  Whether serious
>> or not.
>>
>> Does this person need directions to some local mental healthcare
>> resources?
>>
>>
>> On Sat, 9 Jul 2022 at 08:48, Keith Medcalf  wrote:
>>
>>>
>>> >I can't either, but the reality right now seems to be that 911 calls are
>>> >failing for anyone on a Rogers cellphone.
>>>
>>> This is par for the course.  These people chose to deal with Rogers
>>> despite knowing the consequences.  It is like if you bought a Rogers
>>> Snowblower and it did not work.  That would mean that people who bought the
>>> Rogers Snowblower will not be using it to get rid of the snow that is
>>> preventing them from leaving their house.
>>>
>>> Mutatis mutandis when Rogers is down things that are Rogers dependent
>>> will not work.
>>>
>>> Some people are so retarded it is astonishing!
>>>
>>> --
>>> (CAUTION) You are advised that if you attack my person or property, you
>>> will be put down in accordance with the provisions of section 34 & 35 of
>>> the Criminal Code respectively.  If you are brandishing (or in possession)
>>> of a weapon then lethal force will be applied to your person in accordance
>>> with the law.  This means that your misadventures may end in your death.
>>> Consider yourself cautioned and govern your actions appropriately.
>>>
>>> >-Original Message-
>>> >From: NANOG  On Behalf Of
>>> >Eric Kuhnke
>>> >Sent: Friday, 8 July, 2022 13:34
>>> >To: jim deleskie 
>>> >Cc: NANOG list 
>>> >Subject: Re: Rogers Outage Canada
>>> >
>>> >
>>> >I have seen anecdotal reports that the mobile network is in a half
>>> broken
>>> >state that phones remain registered to, so a 911 call will attempt and
>>> >then fail.
>>> >
>>> >
>>> >This is unlike what would happen if you had a US/Canada cellphone with
>>> >battery power but no SIM card in it that would search for any available
>>> >network in RF range for a 911 call if needed.
>>> >
>>> >
>>> >On Fri, 8 Jul 2022 at 12:31, jim deleskie >> ><mailto:deles...@gmail.com> > wrote:
>>> >
>>> >
>>> >   i cant see BGP taking out SS7.
>>> >
>>> >   -jim
>>> >
>>> >   On Fri, Jul 8, 2022 at 2:45 PM Snowmobile2004
>>> >mailto:greenjosh6...@gmail.com> > wrote:
>>> >
>>> >
>>> >   According to Cloudflare Radar
>>> ><https://radar.cloudflare.com/asn/812?date_filter=last_24_hours> ,
>>> Rogers
>>> >BGP announcements spiked massively to levels 536,777% higher than normal
>>> >(343,601 vs 64 normally) just minutes before the outage. I would not be
>>> >surprised if this happened to be the culprit.
>>> >
>>> >   Regards,
>>> >   Josh Green
>>> >
&

Re: Rogers Outage Canada

[Victor Kuarsingh]

We should move on and focus on constructive comments vs. kicking an
operator when they are down.

That said, demeaning comments, no matter what they are or whom they are
targeted at, is not healthy and is not helpful.

regards,

Victor K

On Sat, Jul 9, 2022 at 9:11 PM L F  wrote:

>
> Lest we ALL MOVE ON….
>
>
> Yes he said
>
> “RETARD” =
>
> The delay in processing the current status of a Situation…
>
>
> Lets move on to bigger n scarier Post mortems:
>
>
> Was This or Was this NOT
>
> A CYBERATTACK ???
>
> Done.
>
> Lets deal with REAL threats not PC references.
>
> Time to evaluate n re eval Our NTS!!
>
> Veni vidi vici 2022
>
>
>
>
> On Sat, Jul 9, 2022 at 5:01 PM Eric Kuhnke  wrote:
>
>> Can we have a discussion with the list admins about a list member
>> appending a threat of violence to their outbound emails?  Whether serious
>> or not.
>>
>> Does this person need directions to some local mental healthcare
>> resources?
>>
>>
>> On Sat, 9 Jul 2022 at 08:48, Keith Medcalf  wrote:
>>
>>>
>>> >I can't either, but the reality right now seems to be that 911 calls are
>>> >failing for anyone on a Rogers cellphone.
>>>
>>> This is par for the course.  These people chose to deal with Rogers
>>> despite knowing the consequences.  It is like if you bought a Rogers
>>> Snowblower and it did not work.  That would mean that people who bought the
>>> Rogers Snowblower will not be using it to get rid of the snow that is
>>> preventing them from leaving their house.
>>>
>>> Mutatis mutandis when Rogers is down things that are Rogers dependent
>>> will not work.
>>>
>>> Some people are so retarded it is astonishing!
>>>
>>> --
>>> (CAUTION) You are advised that if you attack my person or property, you
>>> will be put down in accordance with the provisions of section 34 & 35 of
>>> the Criminal Code respectively.  If you are brandishing (or in possession)
>>> of a weapon then lethal force will be applied to your person in accordance
>>> with the law.  This means that your misadventures may end in your death.
>>> Consider yourself cautioned and govern your actions appropriately.
>>>
>>> >-Original Message-
>>> >From: NANOG  On Behalf Of
>>> >Eric Kuhnke
>>> >Sent: Friday, 8 July, 2022 13:34
>>> >To: jim deleskie 
>>> >Cc: NANOG list 
>>> >Subject: Re: Rogers Outage Canada
>>> >
>>> >
>>> >I have seen anecdotal reports that the mobile network is in a half
>>> broken
>>> >state that phones remain registered to, so a 911 call will attempt and
>>> >then fail.
>>> >
>>> >
>>> >This is unlike what would happen if you had a US/Canada cellphone with
>>> >battery power but no SIM card in it that would search for any available
>>> >network in RF range for a 911 call if needed.
>>> >
>>> >
>>> >On Fri, 8 Jul 2022 at 12:31, jim deleskie >> ><mailto:deles...@gmail.com> > wrote:
>>> >
>>> >
>>> >   i cant see BGP taking out SS7.
>>> >
>>> >   -jim
>>> >
>>> >   On Fri, Jul 8, 2022 at 2:45 PM Snowmobile2004
>>> >mailto:greenjosh6...@gmail.com> > wrote:
>>> >
>>> >
>>> >   According to Cloudflare Radar
>>> ><https://radar.cloudflare.com/asn/812?date_filter=last_24_hours> ,
>>> Rogers
>>> >BGP announcements spiked massively to levels 536,777% higher than normal
>>> >(343,601 vs 64 normally) just minutes before the outage. I would not be
>>> >surprised if this happened to be the culprit.
>>> >
>>> >   Regards,
>>> >   Josh Green
>>> >
>>> >   On Fri, Jul 8, 2022 at 2:19 PM Andrew Paolucci via NANOG
>>> >mailto:nanog@nanog.org> > wrote:
>>> >
>>> >
>>> >   In the early hours of the morning around 2-3am
>>> my modem
>>> >got hit with a configuration update that caused a DHCP release that
>>> >wasn't renewed for about two hours, after rollback the connection was
>>> >fine for 3 hours before this network wide outage.
>>> >
>>> >
>>> >   Maybe a failed night time update was attempted
>>> again
>>> &

Re: Rogers Outage Canada

[L F]

Lest we ALL MOVE ON….


Yes he said

“RETARD” =

The delay in processing the current status of a Situation…


Lets move on to bigger n scarier Post mortems:


Was This or Was this NOT

A CYBERATTACK ???

Done.

Lets deal with REAL threats not PC references.

Time to evaluate n re eval Our NTS!!

Veni vidi vici 2022




On Sat, Jul 9, 2022 at 5:01 PM Eric Kuhnke  wrote:

> Can we have a discussion with the list admins about a list member
> appending a threat of violence to their outbound emails?  Whether serious
> or not.
>
> Does this person need directions to some local mental healthcare resources?
>
>
> On Sat, 9 Jul 2022 at 08:48, Keith Medcalf  wrote:
>
>>
>> >I can't either, but the reality right now seems to be that 911 calls are
>> >failing for anyone on a Rogers cellphone.
>>
>> This is par for the course.  These people chose to deal with Rogers
>> despite knowing the consequences.  It is like if you bought a Rogers
>> Snowblower and it did not work.  That would mean that people who bought the
>> Rogers Snowblower will not be using it to get rid of the snow that is
>> preventing them from leaving their house.
>>
>> Mutatis mutandis when Rogers is down things that are Rogers dependent
>> will not work.
>>
>> Some people are so retarded it is astonishing!
>>
>> --
>> (CAUTION) You are advised that if you attack my person or property, you
>> will be put down in accordance with the provisions of section 34 & 35 of
>> the Criminal Code respectively.  If you are brandishing (or in possession)
>> of a weapon then lethal force will be applied to your person in accordance
>> with the law.  This means that your misadventures may end in your death.
>> Consider yourself cautioned and govern your actions appropriately.
>>
>> >-Original Message-
>> >From: NANOG  On Behalf Of
>> >Eric Kuhnke
>> >Sent: Friday, 8 July, 2022 13:34
>> >To: jim deleskie 
>> >Cc: NANOG list 
>> >Subject: Re: Rogers Outage Canada
>> >
>> >
>> >I have seen anecdotal reports that the mobile network is in a half broken
>> >state that phones remain registered to, so a 911 call will attempt and
>> >then fail.
>> >
>> >
>> >This is unlike what would happen if you had a US/Canada cellphone with
>> >battery power but no SIM card in it that would search for any available
>> >network in RF range for a 911 call if needed.
>> >
>> >
>> >On Fri, 8 Jul 2022 at 12:31, jim deleskie > ><mailto:deles...@gmail.com> > wrote:
>> >
>> >
>> >   i cant see BGP taking out SS7.
>> >
>> >   -jim
>> >
>> >   On Fri, Jul 8, 2022 at 2:45 PM Snowmobile2004
>> >mailto:greenjosh6...@gmail.com> > wrote:
>> >
>> >
>> >   According to Cloudflare Radar
>> ><https://radar.cloudflare.com/asn/812?date_filter=last_24_hours> ,
>> Rogers
>> >BGP announcements spiked massively to levels 536,777% higher than normal
>> >(343,601 vs 64 normally) just minutes before the outage. I would not be
>> >surprised if this happened to be the culprit.
>> >
>> >   Regards,
>> >   Josh Green
>> >
>> >   On Fri, Jul 8, 2022 at 2:19 PM Andrew Paolucci via NANOG
>> >mailto:nanog@nanog.org> > wrote:
>> >
>> >
>> >   In the early hours of the morning around 2-3am my
>> modem
>> >got hit with a configuration update that caused a DHCP release that
>> >wasn't renewed for about two hours, after rollback the connection was
>> >fine for 3 hours before this network wide outage.
>> >
>> >
>> >   Maybe a failed night time update was attempted
>> again
>> >during office hours, I've heard daytime guys are still WFH and night
>> >shift is in building.
>> >
>> >
>> >   I expect we'll never get a real explanation.
>> Rogers is
>> >notorious for withholding any type of helpful or technical information.
>> >
>> >
>> >   Sent from my inoperable Rogers Mobile via
>> emergency eSIM.
>> >
>> >
>> >   Regards,
>> >
>> >   Andrew Paolucci
>> >    Original Message 
>> >   On Jul. 8, 2022, 1:48 p.m., Jay Hennigan <
>> j...@west.net
>> ><mailto:j...@west.net> > wrote:
>> >
>> >
>> >   On 7/8/22 07:44, Robert DeVita wrote: >
>> Does anyone
>> >have information on a widespread Rogers outage in Canada. I > have
>> >customers with multiple sites down. There's discussion on the Outages
>> >mailing list. Seems widespread, affecting all services, mobile, voice,
>> >Internet. No cause or ETR posted yet. -- Jay Hennigan - j...@west.net
>> ><mailto:j...@west.net>  Network Engineering - CCIE #7880 503 897-8550 -
>> >WB6RDV
>> >
>> >
>> >
>> >   --
>> >
>> >   Josh Green.
>>
>>
>>
>>
>> --
  Liz ***
416.660.5456

Re: Rogers Outage Canada

[Eric Kuhnke]

Can we have a discussion with the list admins about a list member appending
a threat of violence to their outbound emails?  Whether serious or not.

Does this person need directions to some local mental healthcare resources?


On Sat, 9 Jul 2022 at 08:48, Keith Medcalf  wrote:

>
> >I can't either, but the reality right now seems to be that 911 calls are
> >failing for anyone on a Rogers cellphone.
>
> This is par for the course.  These people chose to deal with Rogers
> despite knowing the consequences.  It is like if you bought a Rogers
> Snowblower and it did not work.  That would mean that people who bought the
> Rogers Snowblower will not be using it to get rid of the snow that is
> preventing them from leaving their house.
>
> Mutatis mutandis when Rogers is down things that are Rogers dependent will
> not work.
>
> Some people are so retarded it is astonishing!
>
> --
> (CAUTION) You are advised that if you attack my person or property, you
> will be put down in accordance with the provisions of section 34 & 35 of
> the Criminal Code respectively.  If you are brandishing (or in possession)
> of a weapon then lethal force will be applied to your person in accordance
> with the law.  This means that your misadventures may end in your death.
> Consider yourself cautioned and govern your actions appropriately.
>
> >-Original Message-
> >From: NANOG  On Behalf Of
> >Eric Kuhnke
> >Sent: Friday, 8 July, 2022 13:34
> >To: jim deleskie 
> >Cc: NANOG list 
> >Subject: Re: Rogers Outage Canada
> >
> >
> >I have seen anecdotal reports that the mobile network is in a half broken
> >state that phones remain registered to, so a 911 call will attempt and
> >then fail.
> >
> >
> >This is unlike what would happen if you had a US/Canada cellphone with
> >battery power but no SIM card in it that would search for any available
> >network in RF range for a 911 call if needed.
> >
> >
> >On Fri, 8 Jul 2022 at 12:31, jim deleskie  ><mailto:deles...@gmail.com> > wrote:
> >
> >
> >   i cant see BGP taking out SS7.
> >
> >   -jim
> >
> >   On Fri, Jul 8, 2022 at 2:45 PM Snowmobile2004
> >mailto:greenjosh6...@gmail.com> > wrote:
> >
> >
> >   According to Cloudflare Radar
> ><https://radar.cloudflare.com/asn/812?date_filter=last_24_hours> , Rogers
> >BGP announcements spiked massively to levels 536,777% higher than normal
> >(343,601 vs 64 normally) just minutes before the outage. I would not be
> >surprised if this happened to be the culprit.
> >
> >   Regards,
> >   Josh Green
> >
> >   On Fri, Jul 8, 2022 at 2:19 PM Andrew Paolucci via NANOG
> >mailto:nanog@nanog.org> > wrote:
> >
> >
> >   In the early hours of the morning around 2-3am my
> modem
> >got hit with a configuration update that caused a DHCP release that
> >wasn't renewed for about two hours, after rollback the connection was
> >fine for 3 hours before this network wide outage.
> >
> >
> >   Maybe a failed night time update was attempted
> again
> >during office hours, I've heard daytime guys are still WFH and night
> >shift is in building.
> >
> >
> >   I expect we'll never get a real explanation.
> Rogers is
> >notorious for withholding any type of helpful or technical information.
> >
> >
> >   Sent from my inoperable Rogers Mobile via
> emergency eSIM.
> >
> >
> >   Regards,
> >
> >   Andrew Paolucci
> >    Original Message 
> >   On Jul. 8, 2022, 1:48 p.m., Jay Hennigan <
> j...@west.net
> ><mailto:j...@west.net> > wrote:
> >
> >
> >   On 7/8/22 07:44, Robert DeVita wrote: >
> Does anyone
> >have information on a widespread Rogers outage in Canada. I > have
> >customers with multiple sites down. There's discussion on the Outages
> >mailing list. Seems widespread, affecting all services, mobile, voice,
> >Internet. No cause or ETR posted yet. -- Jay Hennigan - j...@west.net
> ><mailto:j...@west.net>  Network Engineering - CCIE #7880 503 897-8550 -
> >WB6RDV
> >
> >
> >
> >   --
> >
> >   Josh Green.
>
>
>
>
>

Re: Rogers Outage Canada

[Suresh Ramasubramanian]

Just leaving this (yes, satire site) link here

https://www.thebeaverton.com/2022/07/rogers-proudly-announces-10-service-charge-for-unplug-and-relax-nationwide-outage/<https://www.thebeaverton.com/2022/07/rogers-proudly-announces-10-service-charge-for-unplug-and-relax-nationwide-outage/?fbclid=IwAR2ylrdNYhKLddbQNQ23WdAZT5V8wEPrBSi-JKp32bdaiXVjEHZdfxwfV6A=e=cl#l5eav3aeuldc0yfzwjk>


--srs

From: NANOG  on behalf of Andrew 
Paolucci via NANOG 
Sent: Saturday, July 9, 2022 12:49:09 PM
To: j...@west.net ; nanog@nanog.org 
Subject: Re: Rogers Outage Canada

Normally I'd take offense, but in my experience of endless technician calls, 
hundreds of hours on the phone with support.


I can assure Rogers staff are the utmost incompetent out of any I've seen. Only 
talked to one support tech with them that even knew what DOCSIS was. How can 
teaching employees about the technology they sell not be part of training?


Only reason they are even able to operate right now is reselling Xfinity 
products from Comcast after wasting billions trying to roll their own solutions.


Regards,

Andrew Paolucci






Sent from Proton Mail mobile




 Original Message 
On Jul. 9, 2022, 3:18 p.m., Jay Hennigan < j...@west.net> wrote:

On 7/9/22 09:54, JASON BOTHE via NANOG wrote: > I see the point you’re trying 
to make but using the word retarded in this context is not only dumb in itself 
but offense. Please be more respectful on this list. Shall we take another spin 
on the euphemism treadmill? -- Jay Hennigan - j...@west.net Network Engineering 
- CCIE #7880 503 897-8550 - WB6RDV

Re: Rogers Outage Canada

[Andrew Paolucci via NANOG]

Normally I'd take offense, but in my experience of endless technician calls, 
hundreds of hours on the phone with support.

I can assure Rogers staff are the utmost incompetent out of any I've seen. Only 
talked to one support tech with them that even knew what DOCSIS was. How can 
teaching employees about the technology they sell not be part of training?

Only reason they are even able to operate right now is reselling Xfinity 
products from Comcast after wasting billions trying to roll their own solutions.

Regards,

Andrew Paolucci

Sent from Proton Mail mobile

 Original Message 
On Jul. 9, 2022, 3:18 p.m., Jay Hennigan wrote:

> On 7/9/22 09:54, JASON BOTHE via NANOG wrote: > I see the point you’re trying 
> to make but using the word retarded in this context is not only dumb in 
> itself but offense. Please be more respectful on this list. Shall we take 
> another spin on the euphemism treadmill? -- Jay Hennigan - j...@west.net 
> Network Engineering - CCIE #7880 503 897-8550 - WB6RDV

Re: Rogers Outage Canada

[Jay Hennigan]

On 7/9/22 09:54, JASON BOTHE via NANOG wrote:

I see the point you’re trying to make but using the word retarded in this 
context is not only dumb in itself but offense. Please be more respectful on 
this list.


Shall we take another spin on the euphemism treadmill?

--
Jay Hennigan - j...@west.net
Network Engineering - CCIE #7880
503 897-8550 - WB6RDV

Re: Rogers Outage Canada

[JASON BOTHE via NANOG]

I see the point you’re trying to make but using the word retarded in this 
context is not only dumb in itself but offense. Please be more respectful on 
this list. 

Thanks



> On Jul 9, 2022, at 10:48, Keith Medcalf  wrote:
> 
> 
>> I can't either, but the reality right now seems to be that 911 calls are
>> failing for anyone on a Rogers cellphone.
> 
> This is par for the course.  These people chose to deal with Rogers despite 
> knowing the consequences.  It is like if you bought a Rogers Snowblower and 
> it did not work.  That would mean that people who bought the Rogers 
> Snowblower will not be using it to get rid of the snow that is preventing 
> them from leaving their house.
> 
> Mutatis mutandis when Rogers is down things that are Rogers dependent will 
> not work.
> 
> Some people are so retarded it is astonishing!
> 
> -- 
> (CAUTION) You are advised that if you attack my person or property, you will 
> be put down in accordance with the provisions of section 34 & 35 of the 
> Criminal Code respectively.  If you are brandishing (or in possession) of a 
> weapon then lethal force will be applied to your person in accordance with 
> the law.  This means that your misadventures may end in your death.  Consider 
> yourself cautioned and govern your actions appropriately.
> 
>> -Original Message-
>> From: NANOG  On Behalf Of
>> Eric Kuhnke
>> Sent: Friday, 8 July, 2022 13:34
>> To: jim deleskie 
>> Cc: NANOG list 
>> Subject: Re: Rogers Outage Canada
>> 
>> 
>> I have seen anecdotal reports that the mobile network is in a half broken
>> state that phones remain registered to, so a 911 call will attempt and
>> then fail.
>> 
>> 
>> This is unlike what would happen if you had a US/Canada cellphone with
>> battery power but no SIM card in it that would search for any available
>> network in RF range for a 911 call if needed.
>> 
>> 
>> On Fri, 8 Jul 2022 at 12:31, jim deleskie > <mailto:deles...@gmail.com> > wrote:
>> 
>> 
>>i cant see BGP taking out SS7.
>> 
>>-jim
>> 
>>On Fri, Jul 8, 2022 at 2:45 PM Snowmobile2004
>> mailto:greenjosh6...@gmail.com> > wrote:
>> 
>> 
>>According to Cloudflare Radar
>> <https://radar.cloudflare.com/asn/812?date_filter=last_24_hours> , Rogers
>> BGP announcements spiked massively to levels 536,777% higher than normal
>> (343,601 vs 64 normally) just minutes before the outage. I would not be
>> surprised if this happened to be the culprit.
>> 
>>Regards,
>>Josh Green
>> 
>>On Fri, Jul 8, 2022 at 2:19 PM Andrew Paolucci via NANOG
>> mailto:nanog@nanog.org> > wrote:
>> 
>> 
>>In the early hours of the morning around 2-3am my modem
>> got hit with a configuration update that caused a DHCP release that
>> wasn't renewed for about two hours, after rollback the connection was
>> fine for 3 hours before this network wide outage.
>> 
>> 
>>Maybe a failed night time update was attempted again
>> during office hours, I've heard daytime guys are still WFH and night
>> shift is in building.
>> 
>> 
>>I expect we'll never get a real explanation. Rogers is
>> notorious for withholding any type of helpful or technical information.
>> 
>> 
>>Sent from my inoperable Rogers Mobile via emergency eSIM.
>> 
>> 
>>Regards,
>> 
>>Andrew Paolucci
>> Original Message 
>>On Jul. 8, 2022, 1:48 p.m., Jay Hennigan < j...@west.net
>> <mailto:j...@west.net> > wrote:
>> 
>> 
>>On 7/8/22 07:44, Robert DeVita wrote: > Does anyone
>> have information on a widespread Rogers outage in Canada. I > have
>> customers with multiple sites down. There's discussion on the Outages
>> mailing list. Seems widespread, affecting all services, mobile, voice,
>> Internet. No cause or ETR posted yet. -- Jay Hennigan - j...@west.net
>> <mailto:j...@west.net>  Network Engineering - CCIE #7880 503 897-8550 -
>> WB6RDV
>> 
>> 
>> 
>>--
>> 
>>Josh Green.
> 
> 
> 
> 

RE: Rogers Outage Canada

[Keith Medcalf]

>I can't either, but the reality right now seems to be that 911 calls are
>failing for anyone on a Rogers cellphone.

This is par for the course.  These people chose to deal with Rogers despite 
knowing the consequences.  It is like if you bought a Rogers Snowblower and it 
did not work.  That would mean that people who bought the Rogers Snowblower 
will not be using it to get rid of the snow that is preventing them from 
leaving their house.

Mutatis mutandis when Rogers is down things that are Rogers dependent will not 
work.

Some people are so retarded it is astonishing!

--
(CAUTION) You are advised that if you attack my person or property, you will be 
put down in accordance with the provisions of section 34 & 35 of the Criminal 
Code respectively.  If you are brandishing (or in possession) of a weapon then 
lethal force will be applied to your person in accordance with the law.  This 
means that your misadventures may end in your death.  Consider yourself 
cautioned and govern your actions appropriately.

>-Original Message-
>From: NANOG  On Behalf Of
>Eric Kuhnke
>Sent: Friday, 8 July, 2022 13:34
>To: jim deleskie 
>Cc: NANOG list 
>Subject: Re: Rogers Outage Canada
>
>
>I have seen anecdotal reports that the mobile network is in a half broken
>state that phones remain registered to, so a 911 call will attempt and
>then fail.
>
>
>This is unlike what would happen if you had a US/Canada cellphone with
>battery power but no SIM card in it that would search for any available
>network in RF range for a 911 call if needed.
>
>
>On Fri, 8 Jul 2022 at 12:31, jim deleskie <mailto:deles...@gmail.com> > wrote:
>
>
>   i cant see BGP taking out SS7.
>
>   -jim
>
>   On Fri, Jul 8, 2022 at 2:45 PM Snowmobile2004
>mailto:greenjosh6...@gmail.com> > wrote:
>
>
>   According to Cloudflare Radar
><https://radar.cloudflare.com/asn/812?date_filter=last_24_hours> , Rogers
>BGP announcements spiked massively to levels 536,777% higher than normal
>(343,601 vs 64 normally) just minutes before the outage. I would not be
>surprised if this happened to be the culprit.
>
>   Regards,
>   Josh Green
>
>   On Fri, Jul 8, 2022 at 2:19 PM Andrew Paolucci via NANOG
>mailto:nanog@nanog.org> > wrote:
>
>
>   In the early hours of the morning around 2-3am my modem
>got hit with a configuration update that caused a DHCP release that
>wasn't renewed for about two hours, after rollback the connection was
>fine for 3 hours before this network wide outage.
>
>
>   Maybe a failed night time update was attempted again
>during office hours, I've heard daytime guys are still WFH and night
>shift is in building.
>
>
>   I expect we'll never get a real explanation. Rogers is
>notorious for withholding any type of helpful or technical information.
>
>
>   Sent from my inoperable Rogers Mobile via emergency 
> eSIM.
>
>
>   Regards,
>
>   Andrew Paolucci
>    Original Message 
>   On Jul. 8, 2022, 1:48 p.m., Jay Hennigan < j...@west.net
><mailto:j...@west.net> > wrote:
>
>
>   On 7/8/22 07:44, Robert DeVita wrote: > Does 
> anyone
>have information on a widespread Rogers outage in Canada. I > have
>customers with multiple sites down. There's discussion on the Outages
>mailing list. Seems widespread, affecting all services, mobile, voice,
>Internet. No cause or ETR posted yet. -- Jay Hennigan - j...@west.net
><mailto:j...@west.net>  Network Engineering - CCIE #7880 503 897-8550 -
>WB6RDV
>
>
>
>   --
>
>   Josh Green.

Re: Rogers Outage Canada

[Clayton Zekelman]

The E911 network in Ontario and Quebec is run by Bell Canada.

The PSAPs may have non-emergency lines that are provided by other carriers.


At 03:56 PM 08/07/2022, Andrew Paolucci via NANOG wrote:
I believe the call centers run by law enforcement and municipalities 
in the GTA are mostly homed on single providers, I know some Bell 
and Telus users in certain areas are still able to reach emergency services.



Regards,

Andrew Paolucci







--

Clayton Zekelman
Managed Network Systems Inc. (MNSi)
3363 Tecumseh Rd. E
Windsor, Ontario
N8W 1H4

tel. 519-985-8410
fax. 519-985-8409

Re: Rogers Outage Canada

[Andrew Paolucci via NANOG]

I believe the call centers run by law enforcement and municipalities in the GTA 
are mostly homed on single providers, I know some Bell and Telus users in 
certain areas are still able to reach emergency services.

Regards,

Andrew Paolucci

 Original Message 
On Jul. 8, 2022, 3:39 p.m., Clayton Zekelman wrote:

> Why not? If the SS7 is encapsulated in IP... (think SIGTRAN).
>
> At 03:30 PM 08/07/2022, jim deleskie wrote:
>
>> i cant see BGP taking out SS7.
>>
>> -jim
>
> --
>
> Clayton Zekelman
> Managed Network Systems Inc. (MNSi)
> 3363 Tecumseh Rd. E
> Windsor, Ontario
> N8W 1H4
>
> tel. 519-985-8410
> fax. 519-985-8409

Re: Rogers Outage Canada

[Clayton Zekelman]

Why not?  If the SS7 is encapsulated in IP... (think SIGTRAN).

At 03:30 PM 08/07/2022, jim deleskie wrote:

i cant see BGP taking out SS7.

-jim



--

Clayton Zekelman
Managed Network Systems Inc. (MNSi)
3363 Tecumseh Rd. E
Windsor, Ontario
N8W 1H4

tel. 519-985-8410
fax. 519-985-8409

Re: Rogers Outage Canada

[Pete Baldwin - TCC]

Seems like their external BGP peers are dead.  If their internal network 
is the same, and they carry M2PA etc, or use BGP/MPLS to carry SS7 
traffic, or their softswitches/STPs rely on BGP/MPLS in some way then 
SS7 might exist, but not be functional.


--
Pete Baldwin


On 2022-07-08 15:30, jim deleskie wrote:

i cant see BGP taking out SS7.

-jim

On Fri, Jul 8, 2022 at 2:45 PM Snowmobile2004 > wrote:


According to Cloudflare Radar
,
Rogers BGP announcements spiked massively to levels 536,777% higher
than normal (343,601 vs 64 normally) just minutes before the outage.
I would not be surprised if this happened to be the culprit.

Regards,
Josh Green

On Fri, Jul 8, 2022 at 2:19 PM Andrew Paolucci via NANOG
mailto:nanog@nanog.org>> wrote:

In the early hours of the morning around 2-3am my modem got hit
with a configuration update that caused a DHCP release that
wasn't renewed for about two hours, after rollback the
connection was fine for 3 hours before this network wide outage.


Maybe a failed night time update was attempted again during
office hours, I've heard daytime guys are still WFH and night
shift is in building.


I expect we'll never get a real explanation. Rogers is notorious
for withholding any type of helpful or technical information.


Sent from my inoperable Rogers Mobile via emergency eSIM.


Regards,

Andrew Paolucci
 Original Message 
On Jul. 8, 2022, 1:48 p.m., Jay Hennigan < j...@west.net
> wrote:


On 7/8/22 07:44, Robert DeVita wrote: > Does anyone have
information on a widespread Rogers outage in Canada. I >
have customers with multiple sites down. There's discussion
on the Outages mailing list. Seems widespread, affecting all
services, mobile, voice, Internet. No cause or ETR posted
yet. -- Jay Hennigan - j...@west.net 
Network Engineering - CCIE #7880 503 897-8550 - WB6RDV 




-- 
*Josh Green.*

Re: Rogers Outage Canada

[Eric Kuhnke]

I can't either, but the reality right now seems to be that 911 calls are
failing for anyone on a Rogers cellphone.

I have seen anecdotal reports that the mobile network is in a half broken
state that phones remain registered to, so a 911 call will attempt and then
fail.

This is unlike what would happen if you had a US/Canada cellphone with
battery power but no SIM card in it that would search for any available
network in RF range for a 911 call if needed.

On Fri, 8 Jul 2022 at 12:31, jim deleskie  wrote:

> i cant see BGP taking out SS7.
>
> -jim
>
> On Fri, Jul 8, 2022 at 2:45 PM Snowmobile2004 
> wrote:
>
>> According to Cloudflare Radar
>> , Rogers
>> BGP announcements spiked massively to levels 536,777% higher than normal
>> (343,601 vs 64 normally) just minutes before the outage. I would not be
>> surprised if this happened to be the culprit.
>>
>> Regards,
>> Josh Green
>>
>> On Fri, Jul 8, 2022 at 2:19 PM Andrew Paolucci via NANOG 
>> wrote:
>>
>>> In the early hours of the morning around 2-3am my modem got hit with a
>>> configuration update that caused a DHCP release that wasn't renewed for
>>> about two hours, after rollback the connection was fine for 3 hours before
>>> this network wide outage.
>>>
>>>
>>> Maybe a failed night time update was attempted again during office
>>> hours, I've heard daytime guys are still WFH and night shift is in building.
>>>
>>>
>>> I expect we'll never get a real explanation. Rogers is notorious for
>>> withholding any type of helpful or technical information.
>>>
>>>
>>> Sent from my inoperable Rogers Mobile via emergency eSIM.
>>>
>>>
>>> Regards,
>>>
>>> Andrew Paolucci
>>>  Original Message 
>>> On Jul. 8, 2022, 1:48 p.m., Jay Hennigan < j...@west.net> wrote:
>>>
>>>
>>> On 7/8/22 07:44, Robert DeVita wrote: > Does anyone have information on
>>> a widespread Rogers outage in Canada. I > have customers with multiple
>>> sites down. There's discussion on the Outages mailing list. Seems
>>> widespread, affecting all services, mobile, voice, Internet. No cause or
>>> ETR posted yet. -- Jay Hennigan - j...@west.net Network Engineering -
>>> CCIE #7880 503 897-8550 - WB6RDV
>>>
>>>
>>
>> --
>> *Josh Green.*
>>
>

Re: Rogers Outage Canada

[jim deleskie]

i cant see BGP taking out SS7.

-jim

On Fri, Jul 8, 2022 at 2:45 PM Snowmobile2004 
wrote:

> According to Cloudflare Radar
> , Rogers
> BGP announcements spiked massively to levels 536,777% higher than normal
> (343,601 vs 64 normally) just minutes before the outage. I would not be
> surprised if this happened to be the culprit.
>
> Regards,
> Josh Green
>
> On Fri, Jul 8, 2022 at 2:19 PM Andrew Paolucci via NANOG 
> wrote:
>
>> In the early hours of the morning around 2-3am my modem got hit with a
>> configuration update that caused a DHCP release that wasn't renewed for
>> about two hours, after rollback the connection was fine for 3 hours before
>> this network wide outage.
>>
>>
>> Maybe a failed night time update was attempted again during office hours,
>> I've heard daytime guys are still WFH and night shift is in building.
>>
>>
>> I expect we'll never get a real explanation. Rogers is notorious for
>> withholding any type of helpful or technical information.
>>
>>
>> Sent from my inoperable Rogers Mobile via emergency eSIM.
>>
>>
>> Regards,
>>
>> Andrew Paolucci
>>  Original Message 
>> On Jul. 8, 2022, 1:48 p.m., Jay Hennigan < j...@west.net> wrote:
>>
>>
>> On 7/8/22 07:44, Robert DeVita wrote: > Does anyone have information on a
>> widespread Rogers outage in Canada. I > have customers with multiple sites
>> down. There's discussion on the Outages mailing list. Seems widespread,
>> affecting all services, mobile, voice, Internet. No cause or ETR posted
>> yet. -- Jay Hennigan - j...@west.net Network Engineering - CCIE #7880 503
>> 897-8550 - WB6RDV
>>
>>
>
> --
> *Josh Green.*
>

Re: Rogers Outage Canada

[Eric Kuhnke]

Whatever they did, it has also taken out SS7/PSTN 911 services for many
millions of people.

https://www.cbc.ca/news/business/rogers-outage-cell-mobile-wifi-1.6514373

On Fri, 8 Jul 2022 at 11:44, Snowmobile2004  wrote:

> According to Cloudflare Radar
> , Rogers
> BGP announcements spiked massively to levels 536,777% higher than normal
> (343,601 vs 64 normally) just minutes before the outage. I would not be
> surprised if this happened to be the culprit.
>
> Regards,
> Josh Green
>
> On Fri, Jul 8, 2022 at 2:19 PM Andrew Paolucci via NANOG 
> wrote:
>
>> In the early hours of the morning around 2-3am my modem got hit with a
>> configuration update that caused a DHCP release that wasn't renewed for
>> about two hours, after rollback the connection was fine for 3 hours before
>> this network wide outage.
>>
>>
>> Maybe a failed night time update was attempted again during office hours,
>> I've heard daytime guys are still WFH and night shift is in building.
>>
>>
>> I expect we'll never get a real explanation. Rogers is notorious for
>> withholding any type of helpful or technical information.
>>
>>
>> Sent from my inoperable Rogers Mobile via emergency eSIM.
>>
>>
>> Regards,
>>
>> Andrew Paolucci
>>  Original Message 
>> On Jul. 8, 2022, 1:48 p.m., Jay Hennigan < j...@west.net> wrote:
>>
>>
>> On 7/8/22 07:44, Robert DeVita wrote: > Does anyone have information on a
>> widespread Rogers outage in Canada. I > have customers with multiple sites
>> down. There's discussion on the Outages mailing list. Seems widespread,
>> affecting all services, mobile, voice, Internet. No cause or ETR posted
>> yet. -- Jay Hennigan - j...@west.net Network Engineering - CCIE #7880 503
>> 897-8550 - WB6RDV
>>
>>
>
> --
> *Josh Green.*
>

Re: Rogers Outage Canada

[Snowmobile2004]

According to Cloudflare Radar
, Rogers
BGP announcements spiked massively to levels 536,777% higher than normal
(343,601 vs 64 normally) just minutes before the outage. I would not be
surprised if this happened to be the culprit.

Regards,
Josh Green

On Fri, Jul 8, 2022 at 2:19 PM Andrew Paolucci via NANOG 
wrote:

> In the early hours of the morning around 2-3am my modem got hit with a
> configuration update that caused a DHCP release that wasn't renewed for
> about two hours, after rollback the connection was fine for 3 hours before
> this network wide outage.
>
>
> Maybe a failed night time update was attempted again during office hours,
> I've heard daytime guys are still WFH and night shift is in building.
>
>
> I expect we'll never get a real explanation. Rogers is notorious for
> withholding any type of helpful or technical information.
>
>
> Sent from my inoperable Rogers Mobile via emergency eSIM.
>
>
> Regards,
>
> Andrew Paolucci
>  Original Message 
> On Jul. 8, 2022, 1:48 p.m., Jay Hennigan < j...@west.net> wrote:
>
>
> On 7/8/22 07:44, Robert DeVita wrote: > Does anyone have information on a
> widespread Rogers outage in Canada. I > have customers with multiple sites
> down. There's discussion on the Outages mailing list. Seems widespread,
> affecting all services, mobile, voice, Internet. No cause or ETR posted
> yet. -- Jay Hennigan - j...@west.net Network Engineering - CCIE #7880 503
> 897-8550 - WB6RDV
>
>

-- 
*Josh Green.*

Re: Rogers Outage Canada

[Andrew Paolucci via NANOG]

In the early hours of the morning around 2-3am my modem got hit with a 
configuration update that caused a DHCP release that wasn't renewed for about 
two hours, after rollback the connection was fine for 3 hours before this 
network wide outage.

Maybe a failed night time update was attempted again during office hours, I've 
heard daytime guys are still WFH and night shift is in building.

I expect we'll never get a real explanation. Rogers is notorious for 
withholding any type of helpful or technical information.

Sent from my inoperable Rogers Mobile via emergency eSIM.

Regards,

Andrew Paolucci
 Original Message 
On Jul. 8, 2022, 1:48 p.m., Jay Hennigan wrote:

> On 7/8/22 07:44, Robert DeVita wrote: > Does anyone have information on a 
> widespread Rogers outage in Canada. I > have customers with multiple sites 
> down. There's discussion on the Outages mailing list. Seems widespread, 
> affecting all services, mobile, voice, Internet. No cause or ETR posted yet. 
> -- Jay Hennigan - j...@west.net Network Engineering - CCIE #7880 503 897-8550 
> - WB6RDV

Re: Rogers Outage Canada

[Jay Hennigan]

On 7/8/22 07:44, Robert DeVita wrote:
Does anyone have information on a widespread Rogers outage in Canada. I 
have customers with multiple sites down.


There's discussion on the Outages mailing list. Seems widespread, 
affecting all services, mobile, voice, Internet. No cause or ETR posted 
yet.


--
Jay Hennigan - j...@west.net
Network Engineering - CCIE #7880
503 897-8550 - WB6RDV

Rogers Outage Canada

[Robert DeVita]

Does anyone have information on a widespread Rogers outage in Canada. I have 
customers with multiple sites down.

Thanks

Rob

[cid:image001.jpg@01D892AF.488E0A30]
Robert DeVita​​
CEO and Founder
t: (469) 581-2160
 |
m: (469) 441-8864
e: radev...@mejeticks.com
 |
w: mejeticks.com
a:
3100 Carlisle St
,
Dallas
,
75204
[cid:image002.png@01D892AF.488E0A30]
[cid:image003.png@01D892AF.488E0A30]
[cid:image004.png@01D892AF.488E0A30]