Re: [outages] News item: Blackberry services down worldwide
On 13/10/11 19:56, Jared Mauch wrote: Rebuilding this trust can take some time. I do expect that with the iMessage stuff that was released yesterday (SMS/MMSoIP to email/phone#) many more companies will shift to using that instead as the value of BBM is decreased. I also wonder what the impact of iMessage and others will be on places like hotel networks as the devices camp out longer/more often on the wifi, etc. We observed the impact to a hotel of the NANOG crowd this week (i wonder if there will be lessons learned on the part of lodgenet, etc?) If we talking about iMessage as replace of BBM, that's probably fine, but it's really niche market. I was really expecting them to release that stuff and allow desktop users to chat with idevice and making iMessage s2s(XMPP) compatible, so anyone could chat with idevice, even not supporting all fancy features, but at least dumb texting.
RE: [outages] News item: Blackberry services down worldwide
-Original Message- From: Nikolay Shopik [mailto:sho...@inblock.ru] Sent: 14 October 2011 10:17 To: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide On 13/10/11 19:56, Jared Mauch wrote: Rebuilding this trust can take some time. I do expect that with the iMessage stuff that was released yesterday (SMS/MMSoIP to email/phone#) many more companies will shift to using that instead as the value of BBM is decreased. I also wonder what the impact of iMessage and others will be on places like hotel networks as the devices camp out longer/more often on the wifi, etc. We observed the impact to a hotel of the NANOG crowd this week (i wonder if there will be lessons learned on the part of lodgenet, etc?) If we talking about iMessage as replace of BBM, that's probably fine, but it's really niche market. I was really expecting them to release that stuff and allow desktop users to chat with idevice and making iMessage s2s(XMPP) compatible, so anyone could chat with idevice, even not supporting all fancy features, but at least dumb texting. My iThings camp on WiFi all the time anyway as they are waiting for push updates, checking mail etc. Of course, all these little things add up and add to the total network traffic (and port counts for NAT)so they all take a toll on networks. I agree though, I would have liked to have seen iMessage cross platform. One of the great things about Skype is that I can talk from PeeCee to MAC to iThing to whatever.. -- Leigh __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: [outages] News item: Blackberry services down worldwide
Jared, On Thu, Oct 13, 2011 at 5:56 PM, Jared Mauch ja...@puck.nether.net wrote: Rebuilding this trust can take some time. I do expect that with the iMessage stuff that was released yesterday (SMS/MMSoIP to email/phone#) many more companies will shift to using that instead as the value of BBM is decreased. With iMessage, Apple is following the lead of multi-platform apps such as Viber (integrated voice over ip) and whatsapp (integrated rich texting over ip). Integrated meaning the unique name/key registered in the system's name lookup service is your phone number, so you automagically discover who of all your address book entries have the application. Turning on whatsapp on my 360 contact address book yielded me 10% of my contact list *online* using it. :) Not being multi-vendor/platform, I wonder if iMessage on iPhone is going to reach similar uptake. Being installed from start certainly helps though, but not piggy backing on the phone numbers is a clear strategic error in my opinion (apple IDs are obviously a long long way from being as universal as phone numbers). I tried out whatsapp yesterday on an old Symbian S60 Nokia (N97) and it works great. Only thing I regret is not trying it out sooner. Now, if mobile devices only had ... globally unique and *reachable* IP addresses, you could even envision sending messages/pictures/video directly from your own device to a peer, with no need for bouncing through overloaded centralized bottlenecks, such as is the case with whatsapp (and certainly iMessage as well). There's certainly a business case in there for a legacy-free, bandwidth-optimized, IP only, LTE-network... (read: no [stupid] tunnels) I also wonder what the impact of iMessage and others will be on places like hotel networks as the devices camp out longer/more often on the wifi, etc. We observed the impact to a hotel of the NANOG crowd this week (i wonder if there will be lessons learned on the part of lodgenet, etc?) I know personally I've observed the attwifi ssid expanding to more places (including hilton branded properties) in the past 6 months to offload cellular data. Offloading is wise, indeed. Cheers, Martin
Re: [outages] News item: Blackberry services down worldwide
What I'm not digging about the entire iMessage I turned off my iMessage option and someone else here in the office was trying to send me a txt. From the looks of it the iPhone does not let you pick between wanting to send an iMessage or txt I could be wrong, but his phone was forcing iMessage and of course I was not getting the messages. Little bit of an issue not getting those messages. Carlos Alcantar Race Communications / Race Team Member 101 Haskins Way, So. San Francisco, CA. 94080 Phone: +1 415 376 3314 Fax: +1 650 246 8901 / carlos *at* race.com / http://www.race.com On 10/14/11 11:48 AM, Martin Millnert milln...@gmail.com wrote: Jared, On Thu, Oct 13, 2011 at 5:56 PM, Jared Mauch ja...@puck.nether.net wrote: Rebuilding this trust can take some time. I do expect that with the iMessage stuff that was released yesterday (SMS/MMSoIP to email/phone#) many more companies will shift to using that instead as the value of BBM is decreased. With iMessage, Apple is following the lead of multi-platform apps such as Viber (integrated voice over ip) and whatsapp (integrated rich texting over ip). Integrated meaning the unique name/key registered in the system's name lookup service is your phone number, so you automagically discover who of all your address book entries have the application. Turning on whatsapp on my 360 contact address book yielded me 10% of my contact list *online* using it. :) Not being multi-vendor/platform, I wonder if iMessage on iPhone is going to reach similar uptake. Being installed from start certainly helps though, but not piggy backing on the phone numbers is a clear strategic error in my opinion (apple IDs are obviously a long long way from being as universal as phone numbers). I tried out whatsapp yesterday on an old Symbian S60 Nokia (N97) and it works great. Only thing I regret is not trying it out sooner. Now, if mobile devices only had ... globally unique and *reachable* IP addresses, you could even envision sending messages/pictures/video directly from your own device to a peer, with no need for bouncing through overloaded centralized bottlenecks, such as is the case with whatsapp (and certainly iMessage as well). There's certainly a business case in there for a legacy-free, bandwidth-optimized, IP only, LTE-network... (read: no [stupid] tunnels) I also wonder what the impact of iMessage and others will be on places like hotel networks as the devices camp out longer/more often on the wifi, etc. We observed the impact to a hotel of the NANOG crowd this week (i wonder if there will be lessons learned on the part of lodgenet, etc?) I know personally I've observed the attwifi ssid expanding to more places (including hilton branded properties) in the past 6 months to offload cellular data. Offloading is wise, indeed. Cheers, Martin
RE: [outages] News item: Blackberry services down worldwide
You are correct. The BES uses PSKs to talk to RIM's servers, which then uses them to talk to the devices over the carrier networks. All of this was in complete failure mode until sometime overnight when it appears to have all started flowing again. Someday either Google or Apple will get off their rear ends and roll out an end to end encrypted service that plugs into corporate email/calendar/workgroup services and we can all gladly toss these horrid little devices in the recycle bins where they belong. Jamie -Original Message- From: Joe Abley [mailto:jab...@hopcount.ca] Sent: Wednesday, October 12, 2011 6:06 PM To: Phil Regnauld Cc: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide On 2011-10-12, at 18:02, Phil Regnauld wrote: Joe Abley (jabley) writes: On 2011-10-12, at 13:05, Leigh Porter wrote: Email on my iPhone is working fine.. ;-) The blackberry message service is centralised with a lot of processing intelligence in the core. Messaging services that use the core as a simple transport and shift the processing intelligence to the edge have different, less-dramatic failure modes. This is not the case for corporate customers with dedicated servers, AFAIU. I'm no expert, but my understanding is that at some/most/all traffic between handhelds and a BES, carried from the handheld device through a cellular network, still flows through RIM. Joe
RE: [outages] News item: Blackberry services down worldwide
It's called Microsoft Exchange ActiveSync :) It works with Android, Apple and Microsoft devices. I believe both Lotus and Groupwise have licensed and support it as well. We have a few (but now, very few) blackberry users remaining. They won't let it go until we rip it out of their hands. -Original Message- From: Jamie Bowden [mailto:ja...@photon.com] Sent: Thursday, October 13, 2011 7:36 AM To: Joe Abley Cc: nanog@nanog.org Subject: RE: [outages] News item: Blackberry services down worldwide You are correct. The BES uses PSKs to talk to RIM's servers, which then uses them to talk to the devices over the carrier networks. All of this was in complete failure mode until sometime overnight when it appears to have all started flowing again. Someday either Google or Apple will get off their rear ends and roll out an end to end encrypted service that plugs into corporate email/calendar/workgroup services and we can all gladly toss these horrid little devices in the recycle bins where they belong. Jamie -Original Message- From: Joe Abley [mailto:jab...@hopcount.ca] Sent: Wednesday, October 12, 2011 6:06 PM To: Phil Regnauld Cc: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide On 2011-10-12, at 18:02, Phil Regnauld wrote: Joe Abley (jabley) writes: On 2011-10-12, at 13:05, Leigh Porter wrote: Email on my iPhone is working fine.. ;-) The blackberry message service is centralised with a lot of processing intelligence in the core. Messaging services that use the core as a simple transport and shift the processing intelligence to the edge have different, less-dramatic failure modes. This is not the case for corporate customers with dedicated servers, AFAIU. I'm no expert, but my understanding is that at some/most/all traffic between handhelds and a BES, carried from the handheld device through a cellular network, still flows through RIM. Joe
RE: [outages] News item: Blackberry services down worldwide
Any idea of when Apple's ActiveSync Implementation will close the gap with what BES does? Like maybe having Important message notifications? Categories? Filters? I use an iPhone, but mail handling on it is lacking. -Original Message- From: Matthew Huff [mailto:mh...@ox.com] Sent: Thursday, October 13, 2011 8:44 AM To: 'Jamie Bowden'; 'Joe Abley' Cc: 'nanog@nanog.org' Subject: RE: [outages] News item: Blackberry services down worldwide It's called Microsoft Exchange ActiveSync :) It works with Android, Apple and Microsoft devices. I believe both Lotus and Groupwise have licensed and support it as well. We have a few (but now, very few) blackberry users remaining. They won't let it go until we rip it out of their hands. -Original Message- From: Jamie Bowden [mailto:ja...@photon.com] Sent: Thursday, October 13, 2011 7:36 AM To: Joe Abley Cc: nanog@nanog.org Subject: RE: [outages] News item: Blackberry services down worldwide You are correct. The BES uses PSKs to talk to RIM's servers, which then uses them to talk to the devices over the carrier networks. All of this was in complete failure mode until sometime overnight when it appears to have all started flowing again. Someday either Google or Apple will get off their rear ends and roll out an end to end encrypted service that plugs into corporate email/calendar/workgroup services and we can all gladly toss these horrid little devices in the recycle bins where they belong. Jamie -Original Message- From: Joe Abley [mailto:jab...@hopcount.ca] Sent: Wednesday, October 12, 2011 6:06 PM To: Phil Regnauld Cc: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide On 2011-10-12, at 18:02, Phil Regnauld wrote: Joe Abley (jabley) writes: On 2011-10-12, at 13:05, Leigh Porter wrote: Email on my iPhone is working fine.. ;-) The blackberry message service is centralised with a lot of processing intelligence in the core. Messaging services that use the core as a simple transport and shift the processing intelligence to the edge have different, less-dramatic failure modes. This is not the case for corporate customers with dedicated servers, AFAIU. I'm no expert, but my understanding is that at some/most/all traffic between handhelds and a BES, carried from the handheld device through a cellular network, still flows through RIM. Joe
RE: [outages] News item: Blackberry services down worldwide
Agreed. Had a customer during the timeframe of this week ditch 90 blackberries for iPhone/android devices. He actually sent me a video after BES finished uninstalling and he shut the server down so help me I'm never getting another one of these damn coasters. One user said when they got the phone where is the silly wheelie clicky thing. IT manager said oh no you just touch the screen. I'm told it was like watching an 8 year old with a box of fireworks and matches For those who complain about security on windows mobile, iPhone or android... you can do l2tp vpn and then ActiveSync on top of that over https. Mobile device policies in Exchange for user experience control. Overall much easier than Blackberry, not dependent on someone else's equipment for things like mail delivery and internet browsing, and one less server to care about. -Original Message- From: Matthew Huff [mailto:mh...@ox.com] Sent: Thursday, October 13, 2011 6:44 AM To: 'Jamie Bowden'; 'Joe Abley' Cc: 'nanog@nanog.org' Subject: RE: [outages] News item: Blackberry services down worldwide It's called Microsoft Exchange ActiveSync :) It works with Android, Apple and Microsoft devices. I believe both Lotus and Groupwise have licensed and support it as well. We have a few (but now, very few) blackberry users remaining. They won't let it go until we rip it out of their hands. -Original Message- From: Jamie Bowden [mailto:ja...@photon.com] Sent: Thursday, October 13, 2011 7:36 AM To: Joe Abley Cc: nanog@nanog.org Subject: RE: [outages] News item: Blackberry services down worldwide You are correct. The BES uses PSKs to talk to RIM's servers, which then uses them to talk to the devices over the carrier networks. All of this was in complete failure mode until sometime overnight when it appears to have all started flowing again. Someday either Google or Apple will get off their rear ends and roll out an end to end encrypted service that plugs into corporate email/calendar/workgroup services and we can all gladly toss these horrid little devices in the recycle bins where they belong. Jamie -Original Message- From: Joe Abley [mailto:jab...@hopcount.ca] Sent: Wednesday, October 12, 2011 6:06 PM To: Phil Regnauld Cc: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide On 2011-10-12, at 18:02, Phil Regnauld wrote: Joe Abley (jabley) writes: On 2011-10-12, at 13:05, Leigh Porter wrote: Email on my iPhone is working fine.. ;-) The blackberry message service is centralised with a lot of processing intelligence in the core. Messaging services that use the core as a simple transport and shift the processing intelligence to the edge have different, less-dramatic failure modes. This is not the case for corporate customers with dedicated servers, AFAIU. I'm no expert, but my understanding is that at some/most/all traffic between handhelds and a BES, carried from the handheld device through a cellular network, still flows through RIM. Joe
RE: [outages] News item: Blackberry services down worldwide
Like Blake mentioned, I for one will also be ditching Blackberry devices due to the poor, irregular service which Blackberry users continue to be subject to due to RIM's inability to provide a stable and reliable service. To add further insult to injury, it just simply is unacceptable to be subject to RIM's high service and licensing costs for BES to ultimately rely on a second-rate infrastructure that causes regular 'blackouts'. Time to more to a standalone device that then relies only on the carriers, which in most cases are just as unreliable. None the less, I for one can't justify paying for an 'enterprise service' to subject to incompetence and instability of the provider. These situations simply arise to often with RIM, yet as a service provider they chose to ignore that impact these outages have on their customers in the corporate arena. Real-time communications in the corporate/enterprise world have no become one of the primary methods of communication, due to the technology RIM et al offer. It is indeed a shame that the likes of Apple iOS Google Android are yet to provide features that compete with BlackBerrys, such as encryption etc. (I am not particular clued up with regards to Windows Mobile however...) All of which, to date, are features which are leveraged in terms of justifying the cost of implementing a Blackberry solution. Furthermore, I found RIM's somewhat patronising updates from RIM's CIOs and CEOs quite insulting, particularly when an official statement had already been released stating the issues were 'resolved' to later contradict this statement and simple refer to it as some kind of 'mistake'. Unacceptable, as I am sure many of you would agree. Regards, P. -Original Message- From: Blake T. Pfankuch [mailto:bl...@pfankuch.me] Sent: 13 October 2011 14:08 To: Matthew Huff; 'Jamie Bowden'; 'Joe Abley' Cc: 'nanog@nanog.org' Subject: RE: [outages] News item: Blackberry services down worldwide Agreed. Had a customer during the timeframe of this week ditch 90 blackberries for iPhone/android devices. He actually sent me a video after BES finished uninstalling and he shut the server down so help me I'm never getting another one of these damn coasters. One user said when they got the phone where is the silly wheelie clicky thing. IT manager said oh no you just touch the screen. I'm told it was like watching an 8 year old with a box of fireworks and matches For those who complain about security on windows mobile, iPhone or android... you can do l2tp vpn and then ActiveSync on top of that over https. Mobile device policies in Exchange for user experience control. Overall much easier than Blackberry, not dependent on someone else's equipment for things like mail delivery and internet browsing, and one less server to care about.
Re: [outages] News item: Blackberry services down worldwide
- Original Message - From: Jamie Bowden ja...@photon.com Someday either Google or Apple will get off their rear ends and roll out an end to end encrypted service that plugs into corporate email/calendar/workgroup services and we can all gladly toss these horrid little devices in the recycle bins where they belong. plugI'm fairly sure K-9 does GPG, at least for the email/plug Cheers, -- jr 'just a doco writer' a -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274
Re: [outages] News item: Blackberry services down worldwide
On Thu, Oct 13, 2011 at 11:13 AM, Jay Ashworth j...@baylink.com wrote: - Original Message - From: Jamie Bowden ja...@photon.com Someday either Google or Apple will get off their rear ends and roll out an end to end encrypted service that plugs into corporate email/calendar/workgroup services and we can all gladly toss these horrid little devices in the recycle bins where they belong. plugI'm fairly sure K-9 does GPG, at least for the email/plug plus normal mail + k9 will do TLS on SMTP and IMAP... or they both do with my mail server just fine. (idevices seeem to also do this well enough) It's possible that the 'encryption' comment from Jamie is really about encrypting the actual device... which I believe Android[0] will do, I don't know if idevices do though. -chris 0: http://googleenterprise.blogspot.com/2011/04/putting-android-to-work-for-your.html
Re: [outages] News item: Blackberry services down worldwide
On Oct 13, 2011, at 11:35 AM, Christopher Morrow wrote: It's possible that the 'encryption' comment from Jamie is really about encrypting the actual device... which I believe Android[0] will do, I don't know if idevices do though. I think the big problem is that rev1 of iDevice did not include on-device crypto, and there was a case where they also 'lied' about their crypto capability to the servers. Rebuilding this trust can take some time. I do expect that with the iMessage stuff that was released yesterday (SMS/MMSoIP to email/phone#) many more companies will shift to using that instead as the value of BBM is decreased. I also wonder what the impact of iMessage and others will be on places like hotel networks as the devices camp out longer/more often on the wifi, etc. We observed the impact to a hotel of the NANOG crowd this week (i wonder if there will be lessons learned on the part of lodgenet, etc?) I know personally I've observed the attwifi ssid expanding to more places (including hilton branded properties) in the past 6 months to offload cellular data. - Jared
RE: [outages] News item: Blackberry services down worldwide
-Original Message- From: Christopher Morrow [mailto:morrowc.li...@gmail.com] Sent: Thursday, October 13, 2011 11:36 AM To: Jay Ashworth Cc: NANOG Subject: Re: [outages] News item: Blackberry services down worldwide On Thu, Oct 13, 2011 at 11:13 AM, Jay Ashworth j...@baylink.com wrote: - Original Message - From: Jamie Bowden ja...@photon.com Someday either Google or Apple will get off their rear ends and roll out an end to end encrypted service that plugs into corporate email/calendar/workgroup services and we can all gladly toss these horrid little devices in the recycle bins where they belong. plugI'm fairly sure K-9 does GPG, at least for the email/plug plus normal mail + k9 will do TLS on SMTP and IMAP... or they both do with my mail server just fine. (idevices seeem to also do this well enough) It's possible that the 'encryption' comment from Jamie is really about encrypting the actual device... which I believe Android[0] will do, I don't know if idevices do though. As of 2.3[.x?] (can't remember if it's a sub release that intro'd this), Android devices can be wholly encrypted, though I don't know if they are by default. All these kludges are great on a small scale, but the BES does end to end encryption for transmission, plugs into Exchange, Lotus, Sametime, proxies internal http[s], and lets us manage policies and push out software updates from a central management point. When it works, it's also scalable, which matters when you have thousands of devices to manage. Jamie
RE: [outages] News item: Blackberry services down worldwide
Pierce, Actually with Windows Mobile and Exchange Enterprise, you can force handheld encryption :) -Original Message- From: Pierce Lynch [mailto:p.ly...@netappliant.com] Sent: Thursday, October 13, 2011 8:35 AM To: 'nanog@nanog.org' Subject: RE: [outages] News item: Blackberry services down worldwide Like Blake mentioned, I for one will also be ditching Blackberry devices due to the poor, irregular service which Blackberry users continue to be subject to due to RIM's inability to provide a stable and reliable service. To add further insult to injury, it just simply is unacceptable to be subject to RIM's high service and licensing costs for BES to ultimately rely on a second-rate infrastructure that causes regular 'blackouts'. Time to more to a standalone device that then relies only on the carriers, which in most cases are just as unreliable. None the less, I for one can't justify paying for an 'enterprise service' to subject to incompetence and instability of the provider. These situations simply arise to often with RIM, yet as a service provider they chose to ignore that impact these outages have on their customers in the corporate arena. Real-time communications in the corporate/enterprise world have no become one of the primary methods of communication, due to the technology RIM et al offer. It is indeed a shame that the likes of Apple iOS Google Android are yet to provide features that compete with BlackBerrys, such as encryption etc. (I am not particular clued up with regards to Windows Mobile however...) All of which, to date, are features which are leveraged in terms of justifying the cost of implementing a Blackberry solution. Furthermore, I found RIM's somewhat patronising updates from RIM's CIOs and CEOs quite insulting, particularly when an official statement had already been released stating the issues were 'resolved' to later contradict this statement and simple refer to it as some kind of 'mistake'. Unacceptable, as I am sure many of you would agree. Regards, P. -Original Message- From: Blake T. Pfankuch [mailto:bl...@pfankuch.me] Sent: 13 October 2011 14:08 To: Matthew Huff; 'Jamie Bowden'; 'Joe Abley' Cc: 'nanog@nanog.org' Subject: RE: [outages] News item: Blackberry services down worldwide Agreed. Had a customer during the timeframe of this week ditch 90 blackberries for iPhone/android devices. He actually sent me a video after BES finished uninstalling and he shut the server down so help me I'm never getting another one of these damn coasters. One user said when they got the phone where is the silly wheelie clicky thing. IT manager said oh no you just touch the screen. I'm told it was like watching an 8 year old with a box of fireworks and matches For those who complain about security on windows mobile, iPhone or android... you can do l2tp vpn and then ActiveSync on top of that over https. Mobile device policies in Exchange for user experience control. Overall much easier than Blackberry, not dependent on someone else's equipment for things like mail delivery and internet browsing, and one less server to care about.
Re: NANOG:RE: [outages] News item: Blackberry services down worldwide
Can't but agree with Jamie. The ability to centralize management for all Blackberry users and _force_ them to comply with company policy (it's an investment bank) saved us lot of hassle when, and it happens regularly, people lose their handsets. Otherwise, it would be all unencrypted, unmonitored and unprotected access points to customer's private data. Some of our representatives recently switched to iphones, but nobody from management will ever be allowed anything than a Blackberry. Andrea On 10/13/11 5:55 PM, Jamie Bowden ja...@photon.com wrote: -Original Message- From: Christopher Morrow [mailto:morrowc.li...@gmail.com] Sent: Thursday, October 13, 2011 11:36 AM To: Jay Ashworth Cc: NANOG Subject: Re: [outages] News item: Blackberry services down worldwide On Thu, Oct 13, 2011 at 11:13 AM, Jay Ashworth j...@baylink.com wrote: - Original Message - From: Jamie Bowden ja...@photon.com Someday either Google or Apple will get off their rear ends and roll out an end to end encrypted service that plugs into corporate email/calendar/workgroup services and we can all gladly toss these horrid little devices in the recycle bins where they belong. plugI'm fairly sure K-9 does GPG, at least for the email/plug plus normal mail + k9 will do TLS on SMTP and IMAP... or they both do with my mail server just fine. (idevices seeem to also do this well enough) It's possible that the 'encryption' comment from Jamie is really about encrypting the actual device... which I believe Android[0] will do, I don't know if idevices do though. As of 2.3[.x?] (can't remember if it's a sub release that intro'd this), Android devices can be wholly encrypted, though I don't know if they are by default. All these kludges are great on a small scale, but the BES does end to end encryption for transmission, plugs into Exchange, Lotus, Sametime, proxies internal http[s], and lets us manage policies and push out software updates from a central management point. When it works, it's also scalable, which matters when you have thousands of devices to manage. Jamie
Re: NANOG:RE: [outages] News item: Blackberry services down worldwide
ActiveSync on Android allows corporate to force compliance with security policy and allow remote wipe. User cannot complete the exchange account setup without permitting the controls. If the user doesn't agree their sync isn't enabled. Moreover, if corporate requirements change sync is disabled until you approve again. That seems like it covers all the bases to me. Sent from my Verizon Wireless Phone -Original message- From: Andrea Gozzi m...@vp44.net To: Jamie Bowden ja...@photon.com, Christopher Morrow morrowc.li...@gmail.com, Jay Ashworth j...@baylink.com Cc: NANOG nanog@nanog.org Sent: Thu, Oct 13, 2011 17:02:53 GMT+00:00 Subject: Re: NANOG:RE: [outages] News item: Blackberry services down worldwide Can't but agree with Jamie. The ability to centralize management for all Blackberry users and _force_ them to comply with company policy (it's an investment bank) saved us lot of hassle when, and it happens regularly, people lose their handsets. Otherwise, it would be all unencrypted, unmonitored and unprotected access points to customer's private data. Some of our representatives recently switched to iphones, but nobody from management will ever be allowed anything than a Blackberry. Andrea On 10/13/11 5:55 PM, Jamie Bowden wrote: -Original Message- From: Christopher Morrow [mailto:morrowc.li...@gmail.com] Sent: Thursday, October 13, 2011 11:36 AM To: Jay Ashworth Cc: NANOG Subject: Re: [outages] News item: Blackberry services down worldwide On Thu, Oct 13, 2011 at 11:13 AM, Jay Ashworth wrote: - Original Message - From: Jamie Bowden Someday either Google or Apple will get off their rear ends and roll out an end to end encrypted service that plugs into corporate email/calendar/workgroup services and we can all gladly toss these horrid little devices in the recycle bins where they belong. I'm fairly sure K-9 does GPG, at least for the email plus normal mail + k9 will do TLS on SMTP and IMAP... or they both do with my mail server just fine. (idevices seeem to also do this well enough) It's possible that the 'encryption' comment from Jamie is really about encrypting the actual device... which I believe Android[0] will do, I don't know if idevices do though. As of 2.3[.x?] (can't remember if it's a sub release that intro'd this), Android devices can be wholly encrypted, though I don't know if they are by default. All these kludges are great on a small scale, but the BES does end to end encryption for transmission, plugs into Exchange, Lotus, Sametime, proxies internal http[s], and lets us manage policies and push out software updates from a central management point. When it works, it's also scalable, which matters when you have thousands of devices to manage. Jamie
Re: [outages] News item: Blackberry services down worldwide
On Oct 13, 2011, at 3:21 PM, McCall, Gabriel wrote: ActiveSync on Android allows corporate to force compliance with security policy and allow remote wipe. User cannot complete the exchange account setup without permitting the controls. If the user doesn't agree their sync isn't enabled. Moreover, if corporate requirements change sync is disabled until you approve again. That seems like it covers all the bases to me. Same on iThings, plus SSL, wipe if 10 incorrect pass codes entered, enforcement of more than a 4-digit PIN pass code, auto-lock timeout, etc., etc. Any device that doesn't do this is likely old and / or going out of biz. I like Jared's attempt to bring this back on topic, though. :) So going down that path, exactly why is iMessage any different from Skype, AIM, Jabber, etc.? I mean other than likely being part of the OS / seamlessly integrated. (I haven't tried it yet, so I am just assuming Apple has done their standard UI magic on this.) In fact, Skype, just as a for instance, is worse on hotel wifi as launching the app on a laptop makes you a middle node for some conversations. Does Skype on $HANDHELD have the same property? -- TTFN, patrick -Original message- From: Andrea Gozzi m...@vp44.net To: Jamie Bowden ja...@photon.com, Christopher Morrow morrowc.li...@gmail.com, Jay Ashworth j...@baylink.com Cc: NANOG nanog@nanog.org Sent: Thu, Oct 13, 2011 17:02:53 GMT+00:00 Subject: Re: NANOG:RE: [outages] News item: Blackberry services down worldwide Can't but agree with Jamie. The ability to centralize management for all Blackberry users and _force_ them to comply with company policy (it's an investment bank) saved us lot of hassle when, and it happens regularly, people lose their handsets. Otherwise, it would be all unencrypted, unmonitored and unprotected access points to customer's private data. Some of our representatives recently switched to iphones, but nobody from management will ever be allowed anything than a Blackberry. Andrea On 10/13/11 5:55 PM, Jamie Bowden wrote: -Original Message- From: Christopher Morrow [mailto:morrowc.li...@gmail.com] Sent: Thursday, October 13, 2011 11:36 AM To: Jay Ashworth Cc: NANOG Subject: Re: [outages] News item: Blackberry services down worldwide On Thu, Oct 13, 2011 at 11:13 AM, Jay Ashworth wrote: - Original Message - From: Jamie Bowden Someday either Google or Apple will get off their rear ends and roll out an end to end encrypted service that plugs into corporate email/calendar/workgroup services and we can all gladly toss these horrid little devices in the recycle bins where they belong. I'm fairly sure K-9 does GPG, at least for the email plus normal mail + k9 will do TLS on SMTP and IMAP... or they both do with my mail server just fine. (idevices seeem to also do this well enough) It's possible that the 'encryption' comment from Jamie is really about encrypting the actual device... which I believe Android[0] will do, I don't know if idevices do though. As of 2.3[.x?] (can't remember if it's a sub release that intro'd this), Android devices can be wholly encrypted, though I don't know if they are by default. All these kludges are great on a small scale, but the BES does end to end encryption for transmission, plugs into Exchange, Lotus, Sametime, proxies internal http[s], and lets us manage policies and push out software updates from a central management point. When it works, it's also scalable, which matters when you have thousands of devices to manage. Jamie
Re: NANOG:RE: [outages] News item: Blackberry services down worldwide
On Thu, Oct 13, 2011 at 12:21 PM, McCall, Gabriel gabriel.mcc...@thyssenkrupp.com wrote: ActiveSync on Android allows corporate to force compliance with security policy and allow remote wipe. User cannot complete the exchange account setup without permitting the controls. If the user doesn't agree their sync isn't enabled. Moreover, if corporate requirements change sync is disabled until you approve again. That seems like it covers all the bases to me. There's two key differences between ActiveSync and BES. The first is that ActiveSync implementations vary widely between different manufacturers/implementations/versions/etc. There is a core set of features that all manufacturers must implement, but it's a very small percentage of the full feature set of controls that ActiveSync supports. Things like enforcing a PIN code fit into this category, but other options like disabling the camera and (from memory) device encryption or even remote wipe are NOT in this category. As a result, even if you enable these features on your Exchange/ActiveSync server, you can't be sure that they are actually being enforced as you can't readily control which devices are being used with ActiveSync, and (realistically) you can't stop a user from changing devices so that even if you gave them a handset that supported all the features you wanted, they could simply move over to a new device that didn't. The second key difference is inbound v's outbound. ActiveSync requires you to allow connections into your network from outside, where BES doesn't. In todays world that's not really an issue - especially as most people will have their email servers accessible from the Internet in some way or other - but in BB's heyday this alone was one of the key differientators for Blackberry v's anything else (be that ActiveSync, POP/IMAP/etc, or any other protocols) With so many companies today working on the entire concept of Mobile Device Management (MDM), Blackberry will fade into insignificance in the not too distant future if they don't come out with something better than the competition - but even today they still allow far better control over handsets than ActiveSync alone does. Scott.
RE: NANOG:RE: [outages] News item: Blackberry services down worldwide
Exchange administration is not my primary job, but in my past experience on Exchange and the iPhone, if I enforced a security policy that the phone could not meet then the user would not be able to sync with the server and setup their account. I remember having to tone back the security policy to a point where the iPhone would actually sync. So effectively they are enforced. You can also simply limit what ActiveSync devices are allowed. If you don't like iPhones but Android is ok, you can do that... at least in Exchange 2010 I can. -Vinny -Original Message- From: Scott Howard [mailto:sc...@doc.net.au] Sent: Thursday, October 13, 2011 5:42 PM To: McCall, Gabriel Cc: NANOG Subject: Re: NANOG:RE: [outages] News item: Blackberry services down worldwide On Thu, Oct 13, 2011 at 12:21 PM, McCall, Gabriel gabriel.mcc...@thyssenkrupp.com wrote: ActiveSync on Android allows corporate to force compliance with security policy and allow remote wipe. User cannot complete the exchange account setup without permitting the controls. If the user doesn't agree their sync isn't enabled. Moreover, if corporate requirements change sync is disabled until you approve again. That seems like it covers all the bases to me. There's two key differences between ActiveSync and BES. The first is that ActiveSync implementations vary widely between different manufacturers/implementations/versions/etc. There is a core set of features that all manufacturers must implement, but it's a very small percentage of the full feature set of controls that ActiveSync supports. Things like enforcing a PIN code fit into this category, but other options like disabling the camera and (from memory) device encryption or even remote wipe are NOT in this category. As a result, even if you enable these features on your Exchange/ActiveSync server, you can't be sure that they are actually being enforced as you can't readily control which devices are being used with ActiveSync, and (realistically) you can't stop a user from changing devices so that even if you gave them a handset that supported all the features you wanted, they could simply move over to a new device that didn't. The second key difference is inbound v's outbound. ActiveSync requires you to allow connections into your network from outside, where BES doesn't. In todays world that's not really an issue - especially as most people will have their email servers accessible from the Internet in some way or other - but in BB's heyday this alone was one of the key differientators for Blackberry v's anything else (be that ActiveSync, POP/IMAP/etc, or any other protocols) With so many companies today working on the entire concept of Mobile Device Management (MDM), Blackberry will fade into insignificance in the not too distant future if they don't come out with something better than the competition - but even today they still allow far better control over handsets than ActiveSync alone does. Scott.
Re: [outages] News item: Blackberry services down worldwide
I have been following this thread for a while and I will have to say I am a tad confused. Remote wipe has been in the iPhone since iOS3.1.3 And if your phone is locked it will wipe after 10 (if I remember correctly) failed unlock attempts. My iPhone communicates completely encrypted. It is set to VPN back to our office. And if we didn't wan't to do that we could could use TLS on our mail to keep that traffic encrypted. But encrypt all is the best approach for us. Personally, I hate mail push. I watch folks in meetings constantly looking down or typing some response and never fully listening to the speakers and not fully engaged in the meeting. Additionally, mail push is indiscriminate and just interrupts my train of thought when I am working. If a communique is truly important whomever can iMessage,SMS,jabber/POTS me; otherwise the mail can just wait till I check my inbox. I understand others feel differently. On an iPhone today you can get push from exchange, iCloud/iMap, Gmail/GCloud, Yahoo, OSX Server (I believe) or set your phone the check every x minutes (after all what could be so important that 15 latency minutes would cause a catastrophe? (During many catastrophe situations sms could take hours or the voice cell network could be tied up and are you that close to whatever to be able to react). If you need instant response... script it. As for filtering, its one of my issues about my iPhone. However, iOS5 supports message flagging and a filter script back on your desktop (where Mail does accept/process message push via IDLE) can flag a message which will sync to your iPhone. Lastly I have never liked RIM's model. It basically inculcates the idea that man in the middle is a good thing which it is not. Just my 2¢ Tom On Oct 13, 2011, at 8:49 AM, Erik Soosalu wrote: Any idea of when Apple's ActiveSync Implementation will close the gap with what BES does? Like maybe having Important message notifications? Categories? Filters? I use an iPhone, but mail handling on it is lacking. -Original Message- From: Matthew Huff [mailto:mh...@ox.com] Sent: Thursday, October 13, 2011 8:44 AM To: 'Jamie Bowden'; 'Joe Abley' Cc: 'nanog@nanog.org' Subject: RE: [outages] News item: Blackberry services down worldwide It's called Microsoft Exchange ActiveSync :) It works with Android, Apple and Microsoft devices. I believe both Lotus and Groupwise have licensed and support it as well. We have a few (but now, very few) blackberry users remaining. They won't let it go until we rip it out of their hands. -Original Message- From: Jamie Bowden [mailto:ja...@photon.com] Sent: Thursday, October 13, 2011 7:36 AM To: Joe Abley Cc: nanog@nanog.org Subject: RE: [outages] News item: Blackberry services down worldwide You are correct. The BES uses PSKs to talk to RIM's servers, which then uses them to talk to the devices over the carrier networks. All of this was in complete failure mode until sometime overnight when it appears to have all started flowing again. Someday either Google or Apple will get off their rear ends and roll out an end to end encrypted service that plugs into corporate email/calendar/workgroup services and we can all gladly toss these horrid little devices in the recycle bins where they belong. Jamie -Original Message- From: Joe Abley [mailto:jab...@hopcount.ca] Sent: Wednesday, October 12, 2011 6:06 PM To: Phil Regnauld Cc: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide On 2011-10-12, at 18:02, Phil Regnauld wrote: Joe Abley (jabley) writes: On 2011-10-12, at 13:05, Leigh Porter wrote: Email on my iPhone is working fine.. ;-) The blackberry message service is centralised with a lot of processing intelligence in the core. Messaging services that use the core as a simple transport and shift the processing intelligence to the edge have different, less-dramatic failure modes. This is not the case for corporate customers with dedicated servers, AFAIU. I'm no expert, but my understanding is that at some/most/all traffic between handhelds and a BES, carried from the handheld device through a cellular network, still flows through RIM. Joe
Re: [outages] News item: Blackberry services down worldwide
On 10/13/11 3:30 PM, Patrick W. Gilmore wrote: In fact, Skype, just as a for instance, is worse on hotel wifi as launching the app on a laptop makes you a middle node for some conversations. Per the Skype IT administrator guide, a Skype node will not become a supernode unless it has a public IP address and meets the memory, bandwidth, and uptime requirements. It will not become a relay node unless it has a public IP address and is directly reachable from the Internet. It is very unlikely that launching the Skype app on a laptop on hotel wi-fi would meet these requirements. Does Skype on $HANDHELD have the same property? Not as far as I know, for the obvious reason that handheld devices have network connections that are suboptimal for this. Matthew Kaufman
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
Guys the outage has moved to U.S and Canada, I think we need to look at this perhaps being sabotage. http://news.cnet.com/8301-30686_3-20119163-266/blackberry-service-issues-spread-to-u.s-and-canada/ Andrew From: Frank Bulk frnk...@iname.com To: outa...@outages.org Sent: Tuesday, October 11, 2011 7:32 PM Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) And continues: “RIM'S SERVICE OUTAGE CONTINUES INTO DAY 2” http://www.channelstv.com/global/news_details.php?nid=29652cat=Politics Frank From:andrew.wallace [mailto:andrew.wall...@rocketmail.com] Sent: Monday, October 10, 2011 2:52 PM To: frnk...@iname.com Cc: outa...@outages.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) RIM shares down as BlackBerry outage continues http://www.marketwatch.com/story/rim-shares-down-as-blackberry-outage-continues-2011-10-10 Andrew From:Frank Bulk frnk...@iname.com To: outa...@outages.org Sent: Monday, October 10, 2011 2:47 PM Subject: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) http://english.ahram.org.eg/NewsContent/3/12/23792/Business/Economy/Blackber ry-services-down-worldwide,-Egypt-affected.aspx FYI ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
What kills me is what they have told the public. The lost a core switch. I don't know if they actually mean network switch or not but I'm pretty sure any of us that work on an enterprise environment know how to factor N+1 just for these types of days. And then the backup solution failed? I'm not buying it either. -Hammer- I was a normal American nerd -Jack Herer On 10/12/2011 09:47 AM, andrew.wallace wrote: Guys the outage has moved to U.S and Canada, I think we need to look at this perhaps being sabotage. http://news.cnet.com/8301-30686_3-20119163-266/blackberry-service-issues-spread-to-u.s-and-canada/ Andrew From: Frank Bulkfrnk...@iname.com To: outa...@outages.org Sent: Tuesday, October 11, 2011 7:32 PM Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) And continues: “RIM'S SERVICE OUTAGE CONTINUES INTO DAY 2” http://www.channelstv.com/global/news_details.php?nid=29652cat=Politics Frank From:andrew.wallace [mailto:andrew.wall...@rocketmail.com] Sent: Monday, October 10, 2011 2:52 PM To: frnk...@iname.com Cc: outa...@outages.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) RIM shares down as BlackBerry outage continues http://www.marketwatch.com/story/rim-shares-down-as-blackberry-outage-continues-2011-10-10 Andrew From:Frank Bulkfrnk...@iname.com To: outa...@outages.org Sent: Monday, October 10, 2011 2:47 PM Subject: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) http://english.ahram.org.eg/NewsContent/3/12/23792/Business/Economy/Blackber ry-services-down-worldwide,-Egypt-affected.aspx FYI ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
Never put down to malice which can be more easily explained by stupidity.. or in this case failure. RIM explained the problem earlier.. The messaging and browsing delays being experienced by BlackBerry users in Europe, the Middle East, Africa, India, Brazil, Chile and Argentina were caused by a core switch failure within RIM's infrastructure. Although the system is designed to failover to a back-up switch, the failover did not function as previously tested. As a result, a large backlog of data was generated and we are now working to clear that backlog and restore normal service as quickly as possible. We apologise for any inconvenience and we will continue to keep you informed. This appears to have been a result of a change on monday The problems began at about 11am on Monday. The Guardian understands that RIM was attempting a software upgrade on its database but suffered corruption problems, and that attempts to switch back to an older version led to a collapse http://www.guardian.co.uk/technology/2011/oct/12/blackberry-outage-executive-apologies?newsfeed=true thanks andrew On Wed, Oct 12, 2011 at 3:47 PM, andrew.wallace andrew.wall...@rocketmail.com wrote: Guys the outage has moved to U.S and Canada, I think we need to look at this perhaps being sabotage. http://news.cnet.com/8301-30686_3-20119163-266/blackberry-service-issues-spread-to-u.s-and-canada/ Andrew From: Frank Bulk frnk...@iname.com To: outa...@outages.org Sent: Tuesday, October 11, 2011 7:32 PM Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) And continues: “RIM'S SERVICE OUTAGE CONTINUES INTO DAY 2” http://www.channelstv.com/global/news_details.php?nid=29652cat=Politics Frank From:andrew.wallace [mailto:andrew.wall...@rocketmail.com] Sent: Monday, October 10, 2011 2:52 PM To: frnk...@iname.com Cc: outa...@outages.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) RIM shares down as BlackBerry outage continues http://www.marketwatch.com/story/rim-shares-down-as-blackberry-outage-continues-2011-10-10 Andrew From:Frank Bulk frnk...@iname.com To: outa...@outages.org Sent: Monday, October 10, 2011 2:47 PM Subject: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) http://english.ahram.org.eg/NewsContent/3/12/23792/Business/Economy/Blackber ry-services-down-worldwide,-Egypt-affected.aspx FYI ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages
RE: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
Maybe they use the same security solutions as Playstation Network does... that would explain a lot suddenly. Paul -Original Message- From: andrew.wallace [mailto:andrew.wall...@rocketmail.com] Sent: Wednesday, October 12, 2011 10:47 AM To: frnk...@iname.com Cc: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) Guys the outage has moved to U.S and Canada, I think we need to look at this perhaps being sabotage. http://news.cnet.com/8301-30686_3-20119163-266/blackberry-service-issues-spread-to-u.s-and-canada/ Andrew From: Frank Bulk frnk...@iname.com To: outa...@outages.org Sent: Tuesday, October 11, 2011 7:32 PM Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) And continues: “RIM'S SERVICE OUTAGE CONTINUES INTO DAY 2” http://www.channelstv.com/global/news_details.php?nid=29652cat=Politics Frank From:andrew.wallace [mailto:andrew.wall...@rocketmail.com] Sent: Monday, October 10, 2011 2:52 PM To: frnk...@iname.com Cc: outa...@outages.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) RIM shares down as BlackBerry outage continues http://www.marketwatch.com/story/rim-shares-down-as-blackberry-outage-continues-2011-10-10 Andrew From:Frank Bulk frnk...@iname.com To: outa...@outages.org Sent: Monday, October 10, 2011 2:47 PM Subject: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) http://english.ahram.org.eg/NewsContent/3/12/23792/Business/Economy/Blackber ry-services-down-worldwide,-Egypt-affected.aspx FYI ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages
Re: [outages] News item: Blackberry services down worldwide
Our blackberry service with Us Cellular in Missouri started having issues about 8am this morning.
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
On 10/12/11 07:47 , andrew.wallace wrote: Guys the outage has moved to U.S and Canada, I think we need to look at this perhaps being sabotage. http://news.cnet.com/8301-30686_3-20119163-266/blackberry-service-issues-spread-to-u.s-and-canada/ North American outages of the blackberry platform (particularly related to upgrades gone wrong) were not uncommon. Think for example sept 10, dec 18 and dec 22 2009. Andrew From: Frank Bulk frnk...@iname.com To: outa...@outages.org Sent: Tuesday, October 11, 2011 7:32 PM Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) And continues: “RIM'S SERVICE OUTAGE CONTINUES INTO DAY 2” http://www.channelstv.com/global/news_details.php?nid=29652cat=Politics Frank From:andrew.wallace [mailto:andrew.wall...@rocketmail.com] Sent: Monday, October 10, 2011 2:52 PM To: frnk...@iname.com Cc: outa...@outages.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) RIM shares down as BlackBerry outage continues http://www.marketwatch.com/story/rim-shares-down-as-blackberry-outage-continues-2011-10-10 Andrew From:Frank Bulk frnk...@iname.com To: outa...@outages.org Sent: Monday, October 10, 2011 2:47 PM Subject: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) http://english.ahram.org.eg/NewsContent/3/12/23792/Business/Economy/Blackber ry-services-down-worldwide,-Egypt-affected.aspx FYI ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
On Wed, 12 Oct 2011 07:47:13 PDT, andrew.wallace said: Guys the outage has moved to U.S and Canada, I think we need to look at this perhaps being sabotage. It ain't sabotage till you rule out misconfigured router. Consider the actual real-world threat models and their likelyhoods: 1) Insufficiently caffienated network engineer - this *NEVER* happens in real life, it's a total Bruce Schneier caliber movie-plot scenario. 2) Somebody sabotaging a RIM router. This is more likely, because there's just *bazillions* of people out there that stand to benefit from a RIM outage (and in fact profit more from an outage than from being able to watch traffic as it goes by). It's just a question of which one of those bazillions did it *this* time. Andrew, you *really* need to learn what the actual failure modes and root causes in real-life production networks are, and draw conclusions from reality, not whatever MI-7 inspired dream world the claim of sabotage came from. pgpBxmKLHmFJ6.pgp Description: PGP signature
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
On Wed, 12 Oct 2011 09:52:02 CDT, -Hammer- said: What kills me is what they have told the public. The lost a core switch. I don't know if they actually mean network switch or not but I'm pretty sure any of us that work on an enterprise environment know how to factor N+1 just for these types of days. And then the backup solution failed? I'm not buying it either. Yeah, and that extra comma in the one config file that didn't make a difference when you tested the failover in the lab *never* makes a difference when it hits in the production network, right? Or they changed the config of the primary and it didn't get propogated just right to the backup, or they had mismatched firmware levels on blades in the blades on the primary and backup switches, so traffic that didn't tickle a bug on the primary blades caused the blade to crash on the backup, or... Anybody on this list who's been around long enough probably has enough We should have had N+2 because the N+1'th device failed too stories to drain *several* pitchers of beer at a good pub... I've even had one case where my butt got *saved* from a ohnosecond-class whoops because the N+1'th device *was* crashed (stomped a config file, it replicated, was able to salvage a copy from a device that didn't replicate because it was down at the time). pgpP55SUQUVfz.pgp Description: PGP signature
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
+1 On Oct 12, 2011 11:51 AM, valdis.kletni...@vt.edu wrote: On Wed, 12 Oct 2011 09:52:02 CDT, -Hammer- said: What kills me is what they have told the public. The lost a core switch. I don't know if they actually mean network switch or not but I'm pretty sure any of us that work on an enterprise environment know how to factor N+1 just for these types of days. And then the backup solution failed? I'm not buying it either. Yeah, and that extra comma in the one config file that didn't make a difference when you tested the failover in the lab *never* makes a difference when it hits in the production network, right? Or they changed the config of the primary and it didn't get propogated just right to the backup, or they had mismatched firmware levels on blades in the blades on the primary and backup switches, so traffic that didn't tickle a bug on the primary blades caused the blade to crash on the backup, or... Anybody on this list who's been around long enough probably has enough We should have had N+2 because the N+1'th device failed too stories to drain *several* pitchers of beer at a good pub... I've even had one case where my butt got *saved* from a ohnosecond-class whoops because the N+1'th device *was* crashed (stomped a config file, it replicated, was able to salvage a copy from a device that didn't replicate because it was down at the time).
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
Idiotberry Envoyé de mon iPhone Le 12 oct. 2011 à 17:55, Charles Mills w3y...@gmail.com a écrit : +1 On Oct 12, 2011 11:51 AM, valdis.kletni...@vt.edu wrote: On Wed, 12 Oct 2011 09:52:02 CDT, -Hammer- said: What kills me is what they have told the public. The lost a core switch. I don't know if they actually mean network switch or not but I'm pretty sure any of us that work on an enterprise environment know how to factor N+1 just for these types of days. And then the backup solution failed? I'm not buying it either. Yeah, and that extra comma in the one config file that didn't make a difference when you tested the failover in the lab *never* makes a difference when it hits in the production network, right? Or they changed the config of the primary and it didn't get propogated just right to the backup, or they had mismatched firmware levels on blades in the blades on the primary and backup switches, so traffic that didn't tickle a bug on the primary blades caused the blade to crash on the backup, or... Anybody on this list who's been around long enough probably has enough We should have had N+2 because the N+1'th device failed too stories to drain *several* pitchers of beer at a good pub... I've even had one case where my butt got *saved* from a ohnosecond-class whoops because the N+1'th device *was* crashed (stomped a config file, it replicated, was able to salvage a copy from a device that didn't replicate because it was down at the time).
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
I think it raises serious questions about RIM's DR strategy if a DB corruption or switch failure or whatever can cause this much outage. 'Surely' RIM have an second site that is independent of the primary (within reason) that they could of flipped to when they realised the DB was borked. If not then any business that relies on them needs to be shouting from the rooftops to get RIM to fix it. Chris. On 12 Oct 2011, at 16:49, valdis.kletni...@vt.edu wrote: On Wed, 12 Oct 2011 09:52:02 CDT, -Hammer- said: What kills me is what they have told the public. The lost a core switch. I don't know if they actually mean network switch or not but I'm pretty sure any of us that work on an enterprise environment know how to factor N+1 just for these types of days. And then the backup solution failed? I'm not buying it either. Yeah, and that extra comma in the one config file that didn't make a difference when you tested the failover in the lab *never* makes a difference when it hits in the production network, right? Or they changed the config of the primary and it didn't get propogated just right to the backup, or they had mismatched firmware levels on blades in the blades on the primary and backup switches, so traffic that didn't tickle a bug on the primary blades caused the blade to crash on the backup, or... Anybody on this list who's been around long enough probably has enough We should have had N+2 because the N+1'th device failed too stories to drain *several* pitchers of beer at a good pub... I've even had one case where my butt got *saved* from a ohnosecond-class whoops because the N+1'th device *was* crashed (stomped a config file, it replicated, was able to salvage a copy from a device that didn't replicate because it was down at the time).
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
I have been witness to N+1 HUMAN failures but never a N+1 hardware failure or system/design failure that warranted questioning the need for N+2. Usually your N+1 failure is (as already referenced) pasting in a bad config that gets replicated or something like that. Not saying the hardware is perfect. It's just that I haven't personally seen a full blown failure like that without human help. Closest example would be an update that wasn't properly vetted in dev/test before migrating to prod. I've seen a few of those that I guess you could blame on the system. Even though the humans could have tested better -Hammer- I was a normal American nerd -Jack Herer On 10/12/2011 10:58 AM, Chris Campbell wrote: I think it raises serious questions about RIM's DR strategy if a DB corruption or switch failure or whatever can cause this much outage. 'Surely' RIM have an second site that is independent of the primary (within reason) that they could of flipped to when they realised the DB was borked. If not then any business that relies on them needs to be shouting from the rooftops to get RIM to fix it. Chris. On 12 Oct 2011, at 16:49, valdis.kletni...@vt.edu wrote: On Wed, 12 Oct 2011 09:52:02 CDT, -Hammer- said: What kills me is what they have told the public. The lost a core switch. I don't know if they actually mean network switch or not but I'm pretty sure any of us that work on an enterprise environment know how to factor N+1 just for these types of days. And then the backup solution failed? I'm not buying it either. Yeah, and that extra comma in the one config file that didn't make a difference when you tested the failover in the lab *never* makes a difference when it hits in the production network, right? Or they changed the config of the primary and it didn't get propogated just right to the backup, or they had mismatched firmware levels on blades in the blades on the primary and backup switches, so traffic that didn't tickle a bug on the primary blades caused the blade to crash on the backup, or... Anybody on this list who's been around long enough probably has enough We should have had N+2 because the N+1'th device failed too stories to drain *several* pitchers of beer at a good pub... I've even had one case where my butt got *saved* from a ohnosecond-class whoops because the N+1'th device *was* crashed (stomped a config file, it replicated, was able to salvage a copy from a device that didn't replicate because it was down at the time).
RE: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
-Original Message- From: -Hammer- [mailto:bhmc...@gmail.com] Sent: 12 October 2011 17:10 To: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) I have been witness to N+1 HUMAN failures but never a N+1 hardware failure or system/design failure that warranted questioning the need for N+2. Usually your N+1 failure is (as already referenced) pasting in a bad config that gets replicated or something like that. Not saying the hardware is perfect. It's just that I haven't personally seen a full blown failure like that without human help. You have not seen VIP2-40s and CEF in action ;-) -- Leigh Porter __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
I have and totally get the point ... -- Michael Gatti cell.949.735.5612 ekim.it...@gmail.com (UTC-8) On Oct 12, 2011, at 9:12 AM, Leigh Porter wrote: -Original Message- From: -Hammer- [mailto:bhmc...@gmail.com] Sent: 12 October 2011 17:10 To: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) I have been witness to N+1 HUMAN failures but never a N+1 hardware failure or system/design failure that warranted questioning the need for N+2. Usually your N+1 failure is (as already referenced) pasting in a bad config that gets replicated or something like that. Not saying the hardware is perfect. It's just that I haven't personally seen a full blown failure like that without human help. You have not seen VIP2-40s and CEF in action ;-) -- Leigh Porter __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: [outages] News item: Blackberry services down worldwide
Haven't received an e-mail on my Blackberry since around 4AM, located in Atlanta. On Wed, Oct 12, 2011 at 11:04 AM, Ben Albee bal...@orscheln.com wrote: Our blackberry service with Us Cellular in Missouri started having issues about 8am this morning.
RE: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
They are out there scrambling, trying to figure out where the truck that hit them came from. The PIO has been told to make up a story. Ralph Brandt Communications Engineer HP Enterprise Services Telephone +1 717.506.0802 FAX +1 717.506.4358 Email ralph.bra...@pateam.com 5095 Ritter Rd Mechanicsburg PA 17055 -Original Message- From: -Hammer- [mailto:bhmc...@gmail.com] Sent: Wednesday, October 12, 2011 10:52 AM To: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) What kills me is what they have told the public. The lost a core switch. I don't know if they actually mean network switch or not but I'm pretty sure any of us that work on an enterprise environment know how to factor N+1 just for these types of days. And then the backup solution failed? I'm not buying it either. -Hammer- I was a normal American nerd -Jack Herer On 10/12/2011 09:47 AM, andrew.wallace wrote: Guys the outage has moved to U.S and Canada, I think we need to look at this perhaps being sabotage. http://news.cnet.com/8301-30686_3-20119163-266/blackberry-service-issues-spread-to-u.s-and-canada/ Andrew From: Frank Bulkfrnk...@iname.com To: outa...@outages.org Sent: Tuesday, October 11, 2011 7:32 PM Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) And continues: “RIM'S SERVICE OUTAGE CONTINUES INTO DAY 2” http://www.channelstv.com/global/news_details.php?nid=29652cat=Politics Frank From:andrew.wallace [mailto:andrew.wall...@rocketmail.com] Sent: Monday, October 10, 2011 2:52 PM To: frnk...@iname.com Cc: outa...@outages.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) RIM shares down as BlackBerry outage continues http://www.marketwatch.com/story/rim-shares-down-as-blackberry-outage-continues-2011-10-10 Andrew From:Frank Bulkfrnk...@iname.com To: outa...@outages.org Sent: Monday, October 10, 2011 2:47 PM Subject: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) http://english.ahram.org.eg/NewsContent/3/12/23792/Business/Economy/Blackber ry-services-down-worldwide,-Egypt-affected.aspx FYI ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
RE: [outages] News item: Blackberry services down worldwide
-Original Message- From: D. Marshall Lemcoe Jr. [mailto:fo...@lemcoe.com] Sent: 12 October 2011 18:01 Cc: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide Haven't received an e-mail on my Blackberry since around 4AM, located in Atlanta. Email on my iPhone is working fine.. ;-) -- Leigh __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: [outages] News item: Blackberry services down worldwide
Found this posting: Blackberry down. Research in Motion (RIM) sent the following e-mail to all clients: To: All Blackberry Clients Please be advised that Research in Motion (RIM) is experiencing world-wide connectivity issues affecting email flow to and from all Blackberries. RIM has not provided an expected time to resolution as of yet. Once we receive notice that the issue is resolved, we will forward that information to you. Thank you, Corporate Information Systems/Mobile Services On Wed, Oct 12, 2011 at 10:05, Leigh Porter leigh.por...@ukbroadband.comwrote: -Original Message- From: D. Marshall Lemcoe Jr. [mailto:fo...@lemcoe.com] Sent: 12 October 2011 18:01 Cc: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide Haven't received an e-mail on my Blackberry since around 4AM, located in Atlanta. Email on my iPhone is working fine.. ;-) -- Leigh __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __ -- steve pirk yensid father... the sleeper has awakened... paul atreides - dune kexp.org member august '09 - Google+ pirk.com
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
- Original Message - From: Valdis Kletnieks valdis.kletni...@vt.edu On Wed, 12 Oct 2011 07:47:13 PDT, andrew.wallace said: Guys the outage has moved to U.S and Canada, I think we need to look at this perhaps being sabotage. It ain't sabotage till you rule out misconfigured router. Andrew, you *really* need to learn what the actual failure modes and root causes in real-life production networks are, and draw conclusions from reality, not whatever MI-7 inspired dream world the claim of sabotage came from. In fairness, Valdis, Andrew did not say this was obviously sabotage. He suggested that that possibility be added to the list of things which the RIM employees tasked with finding a root cause consider. I think the old filtering rule applies here: Once is happenstance. Twice is coincidence. Three times is enemy action. If this turns out to look like it came from 3 or more non-cascading failures, then sabotage will look a little more likely. Cheers, -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
Again. I know those stories are out there. I'm blessed with a lower profile or higher karma. One of the two. digging thru cube to fine wood to knock on -Hammer- I was a normal American nerd -Jack Herer On 10/12/2011 11:53 AM, Mike Gatti wrote: I have and totally get the point ... -- Michael Gatti cell.949.735.5612 ekim.it...@gmail.com (UTC-8) On Oct 12, 2011, at 9:12 AM, Leigh Porter wrote: -Original Message- From: -Hammer- [mailto:bhmc...@gmail.com] Sent: 12 October 2011 17:10 To: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) I have been witness to N+1 HUMAN failures but never a N+1 hardware failure or system/design failure that warranted questioning the need for N+2. Usually your N+1 failure is (as already referenced) pasting in a bad config that gets replicated or something like that. Not saying the hardware is perfect. It's just that I haven't personally seen a full blown failure like that without human help. You have not seen VIP2-40s and CEF in action ;-) -- Leigh Porter __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: [outages] News item: Blackberry services down worldwide
On 2011-10-12, at 13:05, Leigh Porter wrote: Email on my iPhone is working fine.. ;-) The blackberry message service is centralised with a lot of processing intelligence in the core. Messaging services that use the core as a simple transport and shift the processing intelligence to the edge have different, less-dramatic failure modes. No news, here. http://isen.com/stupid.html Joe
Re: [outages] News item: Blackberry services down worldwide
I've always believed that RIM's decision to implement email and other services in this way was a very poor choice that at some point would blow up in their faces. My evil half would say that is was a marketer's rather than an engineer's decision. It's one thing when you are basically the only game in town (as RIM was a few years ago), now it's a completely different scenario. RIM already faces a complicated playground. More high-profile incidents like this one and suddenly people start losing confidence in the service... one thing leads to another... then suddenly you're target for acquisition by a huge corporation. Then things look up again but exactly one year later that huge corporation buries everything you did and you're a page in a history book :-) Good luck to them, Carlos On Wed, Oct 12, 2011 at 3:50 PM, Joe Abley jab...@hopcount.ca wrote: On 2011-10-12, at 13:05, Leigh Porter wrote: Email on my iPhone is working fine.. ;-) The blackberry message service is centralised with a lot of processing intelligence in the core. Messaging services that use the core as a simple transport and shift the processing intelligence to the edge have different, less-dramatic failure modes. No news, here. http://isen.com/stupid.html Joe -- -- = Carlos M. Martinez-Cagnazzo http://www.labs.lacnic.net =
Re: [outages] News item: Blackberry services down worldwide
Joe Abley (jabley) writes: On 2011-10-12, at 13:05, Leigh Porter wrote: Email on my iPhone is working fine.. ;-) The blackberry message service is centralised with a lot of processing intelligence in the core. Messaging services that use the core as a simple transport and shift the processing intelligence to the edge have different, less-dramatic failure modes. This is not the case for corporate customers with dedicated servers, AFAIU. P.
Re: [outages] News item: Blackberry services down worldwide
On 2011-10-12, at 18:02, Phil Regnauld wrote: Joe Abley (jabley) writes: On 2011-10-12, at 13:05, Leigh Porter wrote: Email on my iPhone is working fine.. ;-) The blackberry message service is centralised with a lot of processing intelligence in the core. Messaging services that use the core as a simple transport and shift the processing intelligence to the edge have different, less-dramatic failure modes. This is not the case for corporate customers with dedicated servers, AFAIU. I'm no expert, but my understanding is that at some/most/all traffic between handhelds and a BES, carried from the handheld device through a cellular network, still flows through RIM. Joe
Re: [outages] News item: Blackberry services down worldwide
Joe Abley (jabley) writes: This is not the case for corporate customers with dedicated servers, AFAIU. I'm no expert, but my understanding is that at some/most/all traffic between handhelds and a BES, carried from the handheld device through a cellular network, still flows through RIM. Correct - they need to transit at some point through the RIM servers. http://www.interworks.com/blogs/wlyles/2010/01/14/why-rim-outage-affects-users-corporate-bes That's just wrong on so many levels. Cheers, Phil
Re: [outages] News item: Blackberry services down worldwide
On Wed, Oct 12, 2011 at 6:40 PM, Phil Regnauld regna...@nsrc.org wrote: Correct - they need to transit at some point through the RIM servers. http://www.interworks.com/blogs/wlyles/2010/01/14/why-rim-outage-affects-users-corporate-bes That's just wrong on so many levels. yet... people AND CORPORATIONS still use them... Hrm, one wonders how plain-text-like the traffic is between endpoints? how much data is there that could be used to identify the endpoints? Look, jabley's sure sending lots of traffic to capitan knight... maybe there's something going on in jabley-land? just sayin'!
