I am in favor of 3077 being part of HDFS.
- It is one of several way of configuring HDFS HA (shared storage is an
option or if we extend the BackupNode a little, then it will be another option
for HA.)
- I believe I can use part of 3092/3077 to move journals to the Datanodes
down the road.
- I have a hard time seeing 3077 having independent value outside of HDFS
unless a fair amount of work is added to 3077.
- I do agree with Konstantin that this is fairly complex work; by merging it
in we are adding a fair amount of future maintenance work. I am reviewing the
design and code to see if we can simplify parts of this.
On a separate note - response to some of the comments in this thread.
- Even though I am in favor of this becoming part of HDFS, I don't buy the
argument that HDFS HA should not depend on external projects.
- HDFS HA needs ZK which is an external project. I also think it fine for
folks to use Linux HA - there to be a blueprint for HDFS HA with Linux HA for
those who choose to use it.
I never bought this argument in the case of BK either.
Wrt to BK I feel the HDFS team had done a NIH. I believe we could have worked
with the BK team to add/modify it as needed. The BK team was willing.
We originally started with a simpler design for the journal daemon/node and
kept on finding bugs in the design and then ended up reinventing some of the BK
work.
Yes i have read todd's arguments and have debated this with suresh at great
length - reasonable folks disagree at times )
To summarize - i am in favor of this being part of HDFS and am reviewing the
design and code.
sanjay
On Sep 26, 2012, at 2:32 PM, Stack wrote:
> On Tue, Sep 25, 2012 at 11:21 PM, Konstantin Shvachko
> <[email protected]> wrote:
>> I think this is a great work, Todd.
>> And I think we should not merge it into trunk or other branches.
>> As I suggested earlier on this list I think this should be spinned off
>> as a separate project or a subproject.
>>
>
> I'd be -1 on that.
>
> Users shouldn't have to go elsewhere to get a fix for SPOF.
>
> St.Ack
