Re: A few questions about Hadoop and hard-drive failure handling.

2009-07-24 Thread Aaron Kimball
On Fri, Jul 24, 2009 at 6:48 AM, Steve Loughran wrote: > Ryan Smith wrote: > >> > but you dont want to be the one trying to write something just after >> your >> production cluster lost its namenode data. >> >> Steve, >> >> I wasnt planning on trying to solve something like this in production.

Re: A few questions about Hadoop and hard-drive failure handling.

2009-07-24 Thread Steve Loughran
Ryan Smith wrote: > but you dont want to be the one trying to write something just after your production cluster lost its namenode data. Steve, I wasnt planning on trying to solve something like this in production. I would assume everyone here is a professional and wouldn't even think of some

Re: A few questions about Hadoop and hard-drive failure handling.

2009-07-24 Thread Ryan Smith
> but you dont want to be the one trying to write something just after your production cluster lost its namenode data. Steve, I wasnt planning on trying to solve something like this in production. I would assume everyone here is a professional and wouldn't even think of something like this, but

Re: A few questions about Hadoop and hard-drive failure handling.

2009-07-24 Thread Steve Loughran
Ryan Smith wrote: Todd, excellent info, thank you. I use Ganglia, I will set up nagios though, good idea. Just one clarification on Question 1. What if I actually lose all my master data dirs, and have no back up on the secondary name node, are the data blocks on all the slaves lost in that si

Re: A few questions about Hadoop and hard-drive failure handling.

2009-07-23 Thread Todd Lipcon
On Thu, Jul 23, 2009 at 1:07 PM, Ryan Smith wrote: > Todd, excellent info, thank you. I use Ganglia, I will set up nagios > though, good idea. Just one clarification on Question 1. What if I > actually lose all my master data dirs, and have no back up on the secondary > name node, are the data

Re: A few questions about Hadoop and hard-drive failure handling.

2009-07-23 Thread Ryan Smith
Todd, excellent info, thank you. I use Ganglia, I will set up nagios though, good idea. Just one clarification on Question 1. What if I actually lose all my master data dirs, and have no back up on the secondary name node, are the data blocks on all the slaves lost in that situation? I think Go

Re: A few questions about Hadoop and hard-drive failure handling.

2009-07-23 Thread Todd Lipcon
On Thu, Jul 23, 2009 at 11:56 AM, Ryan Smith wrote: > I was wondering if someone could give me some answers or maybe some > pointers > where to look in the code. All these questions are in the same vein of > hard > drive failure. > > Question 1: If a master (system disks/data) is lost for good, c

A few questions about Hadoop and hard-drive failure handling.

2009-07-23 Thread Ryan Smith
I was wondering if someone could give me some answers or maybe some pointers where to look in the code. All these questions are in the same vein of hard drive failure. Question 1: If a master (system disks/data) is lost for good, can the data on all the slave nodes be recovered? meaning are data