In an operation of this scale you're going to have people who basically live in the datacenters, "hands & eyes" support as it's often called. Their jobs range from just going to aisle/rack/position A/B/C and power cycling a box to rack and stack, etc. Some facilities 24x7, some 8x5, some "lights out".
When you have thousands of servers doing something, losing one is generally a nonevent so this isn't really a critical issue. Additionally, the incremental cost of the iLO/DRAC itself, the cabling, the additional cross connect, and the switchport far outweighs the benefits at this kind of scale. Thanks, Brian Desmond br...@briandesmond.com c - 312.731.3132 From: Damien Solodow [mailto:damien.solo...@harrison.edu] Sent: Wednesday, October 14, 2009 2:10 PM To: NT System Admin Issues Subject: RE: A look at the fully-packed racks inside a Facebook data center facility. Considering those have a "virtual power button" that accomplishes the same thing as pressing the real one, pretty unlikely unless there is a physical power loss. The management cards also have their own network interface which should be on a separate vlan from the servers themselves. But if worst comes to worst, you can have ip managed power units. So just log into the PDU for the rack and reset the appropriately labeled power receptacle. From: Sean Martin [mailto:seanmarti...@gmail.com] Sent: Wednesday, October 14, 2009 2:07 PM To: NT System Admin Issues Subject: Re: A look at the fully-packed racks inside a Facebook data center facility. Hey chuck? Can you reset server "WEB23871? The iLO/DRAC card failed and caused the server to bluescreen." Now what? :) - Sean On Wed, Oct 14, 2009 at 10:03 AM, Sean Rector <sean.rec...@vaopera.org<mailto:sean.rec...@vaopera.org>> wrote: +1 Sean Rector, MCSE From: Damien Solodow [mailto:damien.solo...@harrison.edu<mailto:damien.solo...@harrison.edu>] Sent: Wednesday, October 14, 2009 1:48 PM To: NT System Admin Issues Subject: RE: A look at the fully-packed racks inside a Facebook data center facility. Walk to it? That's what iLo/DRAC are for.. From: Jacob [mailto:ja...@excaliburfilms.com<mailto:ja...@excaliburfilms.com>] Sent: Wednesday, October 14, 2009 1:42 PM To: NT System Admin Issues Subject: RE: A look at the fully-packed racks inside a Facebook data center facility. Hmm.. we have 80 servers in three racks in a room next to me. The web servers are named "WEB01", WEB02", etc.... Hey chuck? Can you reset server "WEB23871"? Okay. Give me 30 minutes to walk to it... From: Sam Cayze [mailto:sam.ca...@rollouts.com<mailto:sam.ca...@rollouts.com>] Sent: Wednesday, October 14, 2009 10:21 AM To: NT System Admin Issues Subject: OT: A look at the fully-packed racks inside a Facebook data center facility. Facebook Now Has 30,000 Servers. 25 Terabytes of Log Data - Daily http://www.datacenterknowledge.com/archives/2009/10/13/facebook-now-has-30000-servers/ Information Technology Manager Virginia Opera Association E-Mail: sean.rec...@vaopera.org<mailto:sean.rec...@vaopera.org> Phone: (757) 213-4548 (direct line) {+} Virginia Opera's 35th Anniversary Season<http://www.vaopera.org/> The One You Love Celebrate with a 2009-2010 Subscription: La Bohème<http://www.vaopera.org/html/currentoperas/opera1.cfm>, The Daughter of the Regiment<http://www.vaopera.org/html/currentoperas/opera2.cfm>, Don Giovanni<http://www.vaopera.org/html/currentoperas/opera3.cfm> and Porgy and BessSM<http://www.vaopera.org/html/currentoperas/opera4.cfm> Visit us online at www.vaopera.org<http://www.vaopera.org/> or call 1-866-OPERA-VA The vision of Virginia Opera is to enrich lives through the powerful integration of music, voice and human drama ________________________________ This e-mail and any attached files are confidential and intended solely for the intended recipient(s). Unless otherwise specified, persons unnamed as recipients may not read, distribute, copy or alter this e-mail. Any views or opinions expressed in this e-mail belong to the author and may not necessarily represent those of Virginia Opera. Although precautions have been taken to ensure no viruses are present, Virginia Opera cannot accept responsibility for any loss or damage that may arise from the use of this e-mail or attachments. {*} ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~