Thanks. I should also mention that most of the gear was still on but we had turned off many VMs on physical servers within the first 2.5 hours, so the CPU and hard drive / io load was around zero on such servers. Most of the servers in the hotter suite had fans running at over 75% vs. about 35% in the cooler suite and ambient temp was down to 32 degrees Celcius within four hours.
-- Erik Levinson CTO, Uberflip 416-900-3830 1183 King Street West, Suite 100 Toronto ON M6K 3C5 www.uberflip.com -----Original Message----- From: "Bryan Tong" <cont...@nullivex.com> Sent: Tuesday, July 9, 2013 11:42pm To: "Erik Levinson" <erik.levin...@uberflip.com> Cc: "NANOG mailing list" <nanog@nanog.org> Subject: Re: What to expect after a cooling failure Hello, In my experience with heating issues the only thing that really degrades quickly in event of overheating are hard drives. If you had them spun down it should be fine. CPU / Memory / Motherboards will be fine. The only other thing I can think of having possible issues are PSU's but if they were powered off should be fine as well. Maybe melted wires but I dont think it was hot enough for that. Thanks On Tue, Jul 9, 2013 at 9:28 PM, Erik Levinson <erik.levin...@uberflip.com>wrote: > As some may know, yesterday 151 Front St suffered a cooling failure after > Enwave's facilities were flooded. > > One of the suites that we're in recovered quickly but the other took much > longer and some of our gear shutdown automatically due to overheating. We > shut down remotely many redundant and non-essential systems in the hotter > suite, and transferred remotely some others to the cooler suite, to ensure > that we had a minimum of all core systems running in the hotter suite. We > waited until the temperatures returned to normal, and brought everything > back online. The entire event lasted from approx 18:45 until 01:15. > Apparently ambient temperature was above 43 degrees Celcius at one point on > the cool side of cabinets in the hotter suite. > > For those who have gone through such events in the past, what can one > expect in terms of long-term impact...should we expect some premature > component failures? Does anyone have any stats to share? > > Thanks > > -- > Erik Levinson > CTO, Uberflip > 416-900-3830 > 1183 King Street West, Suite 100 > Toronto ON M6K 3C5 > www.uberflip.com > > > > -- -------------------- Bryan Tong Nullivex LLC | eSited LLC (507) 298-1624
