Re: data center space
True, but 2ms latency in syncing a backup system is much better than 1 month complete loss of service due to *poor* continuity planning. We all know what the next big threats are (nuclear and/or biological), is it worth the risk that the next (and there will be) event is small enough not to affect an area 65 miles across? Is it worth it to lose billions of dollars every year in order to escape unscathed from some hypothetical future event that may not even affect NY? You have to take a balanced approach to continuity planning. Otherwise, you risk going bankrupt long before there is any big catastrophe. Also, I would say that expecting a terror act to knock out a 65 square mile area is being a bit over pessimistic. Pessimal pessimism at its optimal. --Michael Dillon
Re: data center space
[EMAIL PROTECTED] wrote: You have to take a balanced approach to continuity planning. Otherwise, you risk going bankrupt long before there is any big catastrophe. Also, I would say that expecting a terror act to knock out a 65 square mile area is being a bit over pessimistic. Pessimal pessimism at its optimal. --Michael Dillon If any of you have not done so, I would highly recommend reading Bruce Schneier's book 'Beyond Fear'. The particular scenario that is being described here is what he would call a movie plot scenario, in that while it would make a very good movie, it is not at all likely to happen, and is almost impossible to defend against in any sort of a reasonable fashion. -- Josh Cheney [EMAIL PROTECTED] http://www.joshcheney.com
Re: data center space
Also, I would say that expecting a terror act to knock out a 65 square mile area is being a bit over pessimistic. If any of you have not done so, I would highly recommend reading Bruce Schneier's book 'Beyond Fear'. The particular scenario that is being described here is what he would call a movie plot scenario, in that while it would make a very good movie, it is not at all likely to happen, and is almost impossible to defend against in any sort of a reasonable fashion. About a hundred years ago in a little town on the West Coast, a natural catastrophe killed 3,000. Six years earlier a different natural catastrophe killed 8,000 in a small Gulf Coast town. By these standards the 2986 killed in the biggest terrorist attack on the USA is not that big. As military planners know, it is very, very hard to cause largescale damage, even when you have billions of dollars of equipment and tens of thousands of well-trained people to do the work. Terrorism is not about large scale damage, it is about striking fear into large numbers of people while causing only a small amount of damage. Advance planning can reduce the damage caused by an event. In San Francisco, they reinforce the buildings, bridges and other structures. In Galveston, they tell people to run away days in advance of a hurricane. To mitigate the damage caused by a terror attack, you need to help people understand with their rational minds, that their personal risk is extremely low, that the damage is limited and contained, and that they can defeat the terrorists by remaining calm and rational. During the Katrina incident, a data centre in New Orleans remained operational and on the air because they had done a lot of advance planning. They had stocked up supplies that would be needed including food and water. They approached the situation calmly and rationally whenever unexpected events occurred like armed looters entering the building. They did a darn good job considering their major mistake. For some reason, they didn't expect the city to be practically wiped out for several months so they didn't have a live backup site running in another city. On the other hand, thinking back to pre-Katrina days, what are the chances that they could have convinced customers to pay for the existence of a live backup site in another city? In some ways, Katrina was a movie plot scenario, yet that company still managed to survive the disaster by combining typical data center continuity planning along with survivalist style disaster planning. --Michael Dillon
Re: data center space
On 4/21/06, Jim Popovitch [EMAIL PROTECTED] wrote: Five years after 9/11 you would think that people would have locatedbusiness continuity ops much further away (assuming the businesses arebased in NYC) than NJ.I'm sure that regulations require them to be x miles or in another state.But all things should considered... even thecapability for major catastrophic incident(s) to affect primary and(nearby) secondary sites.It's very unlikely that your business needs to plan for something that affects more than about a 20-mile radius. Events like earthquake or hurricane, or even a nuclear disaster, are fairly localized. For disasters the optimum separation is about 30 miles*, which lets people who are not involved in whatever happens to the primary (the other shifts) staff the alternate in an emergency. Add in cost of fiber, latency, etc., and 30 miles is just about perfect. If your business continuity planning is telling folks anything else, I think perhaps they're not getting what they think.* unless it's just 30 miles further down the eq fault line or hurricane path :-) Local conditions change the rule of thumb as to exact distance/direction. -- Jeff
Re: data center space
Date: Tue, 25 Apr 2006 10:17:51 +0100 From: [EMAIL PROTECTED] You have to take a balanced approach to continuity planning. Otherwise, you risk going bankrupt long before there is any big catastrophe. risk analysis Also, I would say that expecting a terror act to knock out a 65 square mile area is being a bit over pessimistic. Pessimal pessimism at its optimal. Who said terrorism? Come on, people. Let's not turn NANOG-L into an RNC sounding board. I seem to recall a *really big* power outage a few years back. And does the New England area ever have major snowstorms? Each region has plenty of inherent hazards, both natural and not. Let's not limit our concerns to the boogeyman is out to get us, particularly when discussing balance. Eddy -- Everquick Internet - http://www.everquick.net/ A division of Brotsman Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita DO NOT send mail to the following addresses: [EMAIL PROTECTED] -*- [EMAIL PROTECTED] -*- [EMAIL PROTECTED] Sending mail to spambait addresses is a great way to get blocked. Ditto for broken OOO autoresponders and foolish AV software backscatter.
RE: data center space
LD Date: Tue, 25 Apr 2006 09:43:51 +1000 LD From: Lincoln Dale LD I suggest you talk to some of the folks you work with that have to LD deal with synchronous replication. LD LD In the world of storage networking synchronous I/O, typically LD anything higher than 1 msec round-trip latency is too high. This is a big can of worms that's probably OT for NANOG -- not to mention likely outside most readers' realm of experience. It _is_ an interesting field, though. I recommend the Morgan Kauffman book series as a good introduction. One also could argue the necessity and sufficiency of synchronous I/O in and of itself. There's a good deal of work, both published and strictly in the lab dealing with transactional commit mechanisms. Eddy -- Everquick Internet - http://www.everquick.net/ A division of Brotsman Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita DO NOT send mail to the following addresses: [EMAIL PROTECTED] -*- [EMAIL PROTECTED] -*- [EMAIL PROTECTED] Sending mail to spambait addresses is a great way to get blocked. Ditto for broken OOO autoresponders and foolish AV software backscatter.
Re: data center space
Five years after 9/11 you would think that people would have located business continuity ops much further away (assuming the businesses are based in NYC) than NJ. The financial industry has to have their NY backups somewhere else in the NY area because they generally require proximity to the NYSE. The latency from the NYSE to their office is more important than anything else. Also, there are many other stock markets outside New York in London, Frankfurt, Tokyo, etc. There are also ECNs which are electronic trading networks that are not reliant on a single physical location. There is redundancy at all layers. The NYSE has its own redundancy, all the brokerage firms have their redundancy, the banks have theirs, the clearing companies have theirs, and so on. Even within a single organization they likely have different backup locations for servers, for data backups, for office space, etc. Lots of people have been working on this for quite a few years now and they generally hedge their bets by offsetting the high risk backup location in Jersey City with some lower risk backup sites further away. It's all about risk management, a world in which there are few absolutes. --Michael Dillon
RE: data center space
Five years after 9/11 you would think that people would have located business continuity ops much further away (assuming the businesses are [..] Disclaimer: I work for someone who provides outsourcing services including the area of business continuity. I suggest you talk to some of the folks you work with that have to deal with synchronous replication. In the world of storage networking synchronous I/O, typically anything higher than 1 msec round-trip latency is too high.
Re: data center space
Lincoln Dale wrote: I suggest you talk to some of the folks you work with that have to deal with synchronous replication. In the world of storage networking synchronous I/O, typically anything higher than 1 msec round-trip latency is too high. True, but 2ms latency in syncing a backup system is much better than 1 month complete loss of service due to *poor* continuity planning. We all know what the next big threats are (nuclear and/or biological), is it worth the risk that the next (and there will be) event is small enough not to affect an area 65 miles across? -Jim P.
RE: data center space
Lincoln Dale wrote: I suggest you talk to some of the folks you work with that have to deal with synchronous replication. In the world of storage networking synchronous I/O, typically anything higher than 1 msec round-trip latency is too high. True, but 2ms latency in syncing a backup system is much better than 1 month complete loss of service due to *poor* continuity planning. We all know what the next big threats are (nuclear and/or biological), is it worth the risk that the next (and there will be) event is small enough not to affect an area 65 miles across? Once again, I suggest you talk to the folks you work with that deal with replication. My experience is that large NY financials do both sync replication for 90 miles and then async replication to a third tertiary location that is 200+ miles away. Not sure I agree with your on where you think the next big threats are .. but I don't think we could discuss that with any signal:noise ratio! cheers, lincoln.
Re: data center space
On Mon, 24 Apr 2006 22:43:47 EDT, Jim Popovitch said: all know what the next big threats are (nuclear and/or biological), is it worth the risk that the next (and there will be) event is small enough not to affect an area 65 miles across? If I was a Manhattan based company, and an event took out everything within 65 miles of there, whether my backups are OK is suddenly going to be the *least* of my concerns. (Hint - what percent of your employees commute *less* than 65 miles to work, and what disaster-recovery plans do you have for *them*?) pgpbZkm3vdLBZ.pgp Description: PGP signature
Re: data center space
On Tue, Apr 18, 2006 at 09:34:41AM -0700, Philip Lavine wrote: Can someone tell me if I am out of luck. I am trying to get a 10x10 cage in New Jersey (Jersey City area) but it seems everybody is at capacity. What happened? My guess (this being NJ) is an aftereffect of the 9/11/2001 disaster. By five years after, most companies who could be affected by such an outage may have relocated a continuing-operations set of machines to one or more colo data centers. I don't know why the data centers would not have expanded to meet the influx, though. -- Joe Yao --- This message is not an official statement of OSIS Center policies.
Re: data center space
At 06:51 AM 4/21/2006, you wrote: On Tue, Apr 18, 2006 at 09:34:41AM -0700, Philip Lavine wrote: Can someone tell me if I am out of luck. I am trying to get a 10x10 cage in New Jersey (Jersey City area) but it seems everybody is at capacity. What happened? My guess (this being NJ) is an aftereffect of the 9/11/2001 disaster. By five years after, most companies who could be affected by such an outage may have relocated a continuing-operations set of machines to one or more colo data centers. I don't know why the data centers would not have expanded to meet the influx, though. I think most of us have expanded. :) I know Focal/Broadwing has space in Jersey City at 1 Evertrust Plaza. Joe, I know you aren't the original poster, but I'm hoping he or she is still reading this thread too. -Robert Tellurian Networks - The Ultimate Internet Connection http://www.tellurian.com | 888-TELLURIAN | 973-300-9211 Well done is better than well said. - Benjamin Franklin
Re: data center space
Joseph S D Yao wrote: On Tue, Apr 18, 2006 at 09:34:41AM -0700, Philip Lavine wrote: Can someone tell me if I am out of luck. I am trying to get a 10x10 cage in New Jersey (Jersey City area) but it seems everybody is at capacity. What happened? My guess (this being NJ) is an aftereffect of the 9/11/2001 disaster. By five years after, most companies who could be affected by such an outage may have relocated a continuing-operations set of machines to one or more colo data centers. I don't know why the data centers would not have expanded to meet the influx, though. Five years after 9/11 you would think that people would have located business continuity ops much further away (assuming the businesses are based in NYC) than NJ. I'm sure that regulations require them to be x miles or in another state. But all things should considered... even the capability for major catastrophic incident(s) to affect primary and (nearby) secondary sites. I think the reasons are probably due to companies/governments thinking (hoping?) that in the event of a catastrophic event the business would be able to get ppl from site A to site B. To me it is ridiculous to assume that anyone would be left at site A, or even in the vicinity of site A. And if they are still around site A after a catastrophic event, would they behave normally and could they be counted on (families, fears, trauma, etc)? I'm an employee, but in desperate times my family comes first (that is a no brainer decision that every CIO should think about). Put your major data/ops centers on different continents, or at least on different coasts. Not big enough to do that? Outsource to someone who is. Don't want to spend the money? Partner with a non-competing similar business that is strategically located away from yours. Don't do the minimum to insure your business survival, do the maximum. Disclaimer: I work for someone who provides outsourcing services including the area of business continuity. -Jim P.
Re: data center space
At 9:36 PM -0400 4/19/06, Martin Hannigan wrote: Remember when folks thought Exodus was crazy for 220w per square foot? 265w/sqft can just handle today's typical blade server power density (allowing for a reasonable amount of wire management and slightly less than full blade loading). If you look at the densities of newly released blade servers, it is not hard to hit 300w+ / sq ft.The more amusing part is those attempting to cool this via raised floor systems, as it takes 24 to 30 of raised floor to achieve the necessary airflow, and this is only going to get worse. /John
Re: data center space
They claim to be full too, at least from a power perspective. They won't run us more power until the city council aproves them running more power to the building.-jimOn 4/18/06, Mike Sawicki [EMAIL PROTECTED] wrote: On Tue, Apr 18, 2006 at 09:34:41AM -0700, Philip Lavine wrote: Can someone tell me if I am out of luck. I am trying to get a 10x10 cage in New Jersey (Jersey City area) but it seems everybody is at capacity. What happened? Try VZN/MCI Carteret, down the Turnpike about 8 miles.--Mike Sawicki
Re: data center space
At 11:06 AM 4/19/2006, jim bartus wrote: They claim to be full too, at least from a power perspective. They won't run us more power until the city council aproves them running more power to the building. -jim There are likely to be sub leases available from tenants in existing. desirable, colocation and a good way to find them is via brokers. Be aware of a few things that have recently transpired. On many of the public colo houses earnings calls, they told analysts that they are trying to keep contracts to one year so they can raise prices year over year, that power pricing is fluid and many facilities are being expanded both space and environmental, that most locations really are full or being held down by lack of cooling for existing dense rack space. Basically get ready to hold out your wallet. I'd try Internap as a last resort (but a great one). Internap presence is not always facilities based (they own) but they tend to take inventory in good colo, place PNAP's, and sublet adjoining cabinets as Internap space. The catch is that if the facility is out of power, Internap is subjected to the same issues by default. I'm suggesting Internap because they have a good footprint and there's bound to be something somewhere. -M -- Martin Hannigan(c) 617-388-2663 Renesys Corporation(w) 617-395-8574 Member of Technical Staff Network Operations [EMAIL PROTECTED]
Re: data center space
On many of the public colo houses earnings calls, they told analysts that they are trying to keep contracts to one year so they can raise prices year over year, that power pricing is fluid and many facilities are being expanded both space and environmental, that most locations really are full or being held down by lack of cooling for existing dense rack space. Basically get ready to hold out your wallet. Is it that? Or, is it some of these companies no realising that charging $250 for a 20 amp outlet is less than their cost, even three years ago? -- Alex Rubenstein, AR97, K2AHR, [EMAIL PROTECTED], latency, Al Reuben Net Access Corporation, 800-NET-ME-36, http://www.nac.net
Re: data center space
At 08:11 PM 4/19/2006, Alex Rubenstein wrote: On many of the public colo houses earnings calls, they told analysts that they are trying to keep contracts to one year so they can raise prices year over year, that power pricing is fluid and many facilities are being expanded both space and environmental, that most locations really are full or being held down by lack of cooling for existing dense rack space. Basically get ready to hold out your wallet. Is it that? Or, is it some of these companies no realising that charging $250 for a 20 amp outlet is less than their cost, even three years ago? I don't know, but I was selling only measured power in 2001 and people liked it. Granted, power was cheaper, but pay as you go was a good model. You still had to cool to breaker density, but it was nice to have no power risk and I would recommend that anyone who can, should convert to measured power billing. Remember when folks thought Exodus was crazy for 220w per square foot? -M -- Martin Hannigan(c) 617-388-2663 Renesys Corporation(w) 617-395-8574 Member of Technical Staff Network Operations [EMAIL PROTECTED]
RE: data center space
Marty Said... At 08:11 PM 4/19/2006, Alex Rubenstein wrote: On many of the public colo houses earnings calls, they told analysts that they are trying to keep contracts to one year so they can raise prices year over year, that power pricing is fluid and many facilities are being expanded both space and environmental, that most locations really are full or being held down by lack of cooling for existing dense rack space. Basically get ready to hold out your wallet. Well, Peter Van Camp of Equinix was asked this during the (extremely informative) Equinix call for Q1 and said that many contracts being signed are still 2-year. The analyst who asked it made the (correct) assertion that shorter contracts are more advantageous now (for IDC providers), considering the tight data center market. And the market is really tight, especially in particular areas. I expect to see many more NANOG postings where can I find good colocation in area X over the next year. Of course, the colocation companies must raise their prices - for one thing, many folks got sweetheart deals during the lean years. For another, energy prices are way up, as we've all noticed, and IDCs use lots of juice. Finally, its supply and demand. Is it that? Or, is it some of these companies no realising that charging $250 for a 20 amp outlet is less than their cost, even three years ago? I don't know, but I was selling only measured power in 2001 and people liked it. Granted, power was cheaper, but pay as you go was a good model. You still had to cool to breaker density, but it was nice to have no power risk and I would recommend that anyone who can, should convert to measured power billing. If energy prices keep going up, one would think that submetered power would be the wave of the future, so that colos can pass through electricity prices - both direct electrical power consumed by the equipment, and the HVAC needed to dissipate the heat. The move to super-dense server platforms is a real killer. Anyone know of many colos currently submetering individual tenants? Remember when folks thought Exodus was crazy for 220w per square foot? Well, in hindsight that aspect of their plan was visionary. I don't suppose if anyone knows if the Exodus designed were seeking to future-proof in general, anticipated these dense server platforms, or just wanted to build more bigger? - Dan BTW, for the folks who like this stuff, there will hopefully be some great datacenter-related talks at NANOG this time, thanks to Josh Snowhorn. Its not too late to make a submission... :)
data center space
Can someone tell me if I am out of luck. I am trying to get a 10x10 cage in New Jersey (Jersey City area) but it seems everybody is at capacity. What happened?
Re: data center space
On Tue, Apr 18, 2006 at 09:34:41AM -0700, Philip Lavine wrote: Can someone tell me if I am out of luck. I am trying to get a 10x10 cage in New Jersey (Jersey City area) but it seems everybody is at capacity. What happened? Try VZN/MCI Carteret, down the Turnpike about 8 miles. -- Mike Sawicki
