Isn't one of the main suppositions of Hadoop that you're using commodity
computing hardware that's not the most reliable, and that it's to be expected
that nodes may go away without notice? Why do you need a high-availability
clustering solution for this?
If you're trying to confront the problem of information exchange (for example,
configuration or job data), you might consider a distributed hash table.
- Conor
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andrew Beekhof
Sent: Thursday, November 06, 2008 4:02 AM
To: General Linux-HA mailing list
Subject: Re: [Linux-HA] OCF for hadoop
On Wed, Nov 5, 2008 at 16:36, Edward Capriolo <[EMAIL PROTECTED]> wrote:
> Hello all,
>
> Many of you may have heard of Apache Hadoop. Hadoop is a distributed
> computing and storage system. Currently hadoop documentation uses SSH
> keys and shell scripts to start components across the clusters. I
> though an OCF script would be an ideal replacement.
>
> I know the SPEC for linux-ha is to run up to 16 nodes. Are larger
> clusters practical? The Hadoop resource would not flip flop much, but
> clusters have been known to be in the thousands, the target is 10,000
>
> What do you guys think?
I think that may be pushing things a bit (in terms of the CIB size).
But I'm quite motivated to improve our scalability... so let me know
if you're interested in pushing the limits
_______________________________________________
Linux-HA mailing list
[email protected]
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
_______________________________________________
Linux-HA mailing list
[email protected]
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
