On Fri, 16 Jun 2006, Matthew Crocker wrote:
I wonder just how much power it takes to cool 450,000 servers.
450,000 servers * 100 Watts/Server = 45,000,000 watts / 3.413 watts/BTU =
13.1 Million BTU / 12000 BTU/Ton = 1100 Tons of cooling
Error: you MULTIPLY 3.413 to go from watts to BTU, not divide. It's be
more like 154,000,000 BTU, /12000 or 12,798 tons.
Also at 100 watts, you are assuming Celerons with single hard drives. We
see more like 120 to 240 depending on config. 100 would be low.
A 30 Ton Liebert system runs about 80 amps @ 480 volts or 38400 watts,
you'll need at least 40 or them to cool 1100 tons which is 1536 Kw * 24 hours
* 7 days * 4.3 weeks = 1,110,000 KwH/month * $0.10/KwH = $111,000 /month in
cooling.
80 amps @ 480 is 80 * 480 * 1.73, or 66 kw. However, they don't draw that
much. A 30 ton unit, worst case (115 degrees outside across the condensor)
will be about 50 kw, assuming you do not have humidification or reheats
turned on.
Second issue: you are assuming 100% cooling efficiency, or, in other
words, that you'd have perfect airflow, perfect air return, etc. Never
happens, especially when you have customers who are idiots.
Third issue: you are assuming there is no heat loss or gain in the
structure of the building. This could be very significant. Let's assume
it's not.
It's likely in an environment like this, you'd have more like 14000 tons.
14000 / 30 = 466 units, @ 50 kw/unit, 23,300,000 watts, / 1000 * 24 *
30.4375 (avg days in a month) = 17,020,000 kw-hrs, @ $0.12 (more likely
with todays fuel prices unless you are in Kentucky) $2,042,400/month.
Also, don't forget the original 450,000 servers at 100 watts (45 mw) would
be $3,944,700/month in power. Also, 450,000 1U servers at 40/rack would be
11,250 racks, which at 10 sq-ft a rack would be 112,000 sq-ft of
datacenter floor space (triple or, more likely, quadruple that for space
for HVAC, generators, switchgear, UPSs, etc). That'd be 500,000 sq-ft at
minimum.
Total is $5,987,000/mon, but you haven't ROIed the millions in electrical
gear (think big: this is about 68 megawatts; $250k/each for a 2 mw
generator (you'd need 40, $10 mm), $50k/each for a 500 kva UPS (you'd need
80 $4mm), millions in panels, breakers, piping, copper wire (700% increase
in copper pricing in the last 24 months, people), etc. Oh, and 466 liebert
30 ton HVAC's, probably $25 to $40k/ea installed ($11 million). Oh, and no
one has installed it yet, and you haven't paid rent on the facility that
will take 2 years to build with probably 100's of workers saleries.
Take $6mm/month, divide by 450,000 servers, $13.33/month/server.
Oh, and 68 Megawatts over 112k ft of floor space is 607 watts/ft. Thats
about 6 times what most centers built in the last couple years are built
at.
But wait, there is more. Just a point of comparison -- Oyster Creek
Nuclear Power generation plant, located here on the Jersey Shore, produces
636 megawatts. You'd take one-tenth of that capacity -- in a bulding that
would sit on a 10 or 20 acre chunk of land. I put this into the 'unlikely'
category. The substation alone to handle stepping 68 mwatts from
transmission to 480v would be probably 4 acres. And, 68 megawatts of power
at 480 volts 81,888 amps. A typicall 200,000 sq-ft multi-tenant office
building has 1600 amps of service; this would be the equivalent of 50
buildings.
Having fun yet?
A 30 ton liebert takes about 30 sq-ft of floor space; 466 of them would be
13,980 sq-ft. If you use a drycooler system, they are about 100 sq-ft, and
youd need 233 of them (60 ton DDNT940's), 23,300 sq-ft of roof space. Each
of those weighs 2,640 pounds, for a total of 615,000 pounds, or 308 tons
(of weight, not HVAC capacity). I won't even spend the CPU cycles figuring
out how many gallons of glycol this would bem but probably a good guess
would be about 50,000 gallons. That'd be about a quarter-million dollars
in glycol.
I'm tired now, time to climb back in my hole. In other words, don't get
me started on the datacenter density issue.
--
Alex Rubenstein, AR97, K2AHR, [EMAIL PROTECTED], latency, Al Reuben
Net Access Corporation, 800-NET-ME-36, http://www.nac.net