[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[
https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13838696#comment-13838696
]
Hadoop QA commented on HBASE-7411:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12589796/7411v4.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 9 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8054//console
This message is automatically generated.
> Use Netflix's Curator zookeeper library
> ---
>
> Key: HBASE-7411
> URL: https://issues.apache.org/jira/browse/HBASE-7411
> Project: HBase
> Issue Type: New Feature
> Components: Zookeeper
>Affects Versions: 0.95.2
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: 7411v2.txt, 7411v2.txt, 7411v3.txt, 7411v4.txt,
> 7411v4.txt, hbase-7411_v0.patch
>
>
> We have mentioned using the Curator library
> (https://github.com/Netflix/curator) elsewhere but we can continue the
> discussion in this.
> The advantages for the curator lib over ours are the recipes. We have very
> similar retrying mechanism, and we don't need much of the nice client-API
> layer.
> We also have similar Listener interface, etc.
> I think we can decide on one of the following options:
> 1. Do not depend on curator. We have some of the recipes, and some custom
> recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991,
> etc). We can also copy / fork some code from there.
> 2. Replace all of our zk usage / connection management to curator. We may
> keep the current set of API's as a thin wrapper.
> 3. Use our own connection management / retry logic, and build a custom
> CuratorFramework implementation for the curator recipes. This will keep the
> current zk logic/code intact, and allow us to use curator-recipes as we see
> fit.
> 4. Allow both curator and our zk layer to manage the connection. We will
> still have 1 connection, but 2 abstraction layers sharing it. This is the
> easiest to implement, but a freak show?
> I have a patch for 4, and now prototyping 2 or 3 whichever will be less
> painful.
> Related issues:
> HBASE-5547
> HBASE-7305
> HBASE-7212
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695779#comment-13695779 ] Enis Soztutar commented on HBASE-7411: -- I've checked it again in Curator. The idempotency guard is there in create() which puts a unique uuid as a prefix of the znode in CreateBuilderImpl (called setProtected()). But, I also could not find the same protected guard in setData() as we do in RecoverableZookeeper. As per our offline discussion, let's not pursue the approach in the patch for using curator only for recipes, but keep the issue open for discussing completely migrating to curator after 0.96 comes out. We can raise the idempotency issue in curator and address it as a prerequisite. > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.95.2 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.95.2 > > Attachments: 7411v2.txt, 7411v2.txt, 7411v3.txt, 7411v4.txt, > 7411v4.txt, hbase-7411_v0.patch > > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13694491#comment-13694491 ] stack commented on HBASE-7411: -- No sure this patch will work [~enis] There are wide gaps during which getZk() returns null. I could wait until recreated but then could be null by the time the client goes to use it. This is because we manage our own zk setup rather than let curator do it? Related, i see no curator recipes to help w/ idempotent operations (the short that recoverablezk writes at the start of all data it writes). > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.95.2 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.95.2 > > Attachments: 7411v2.txt, 7411v2.txt, 7411v3.txt, 7411v4.txt, > 7411v4.txt, hbase-7411_v0.patch > > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[
https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13694332#comment-13694332
]
Hadoop QA commented on HBASE-7411:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12589796/7411v4.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 9 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestHCM
org.apache.hadoop.hbase.security.access.TestAccessController
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.TestZooKeeper.testRegionAssignmentAfterMasterRecoveryDueToZKExpiry(TestZooKeeper.java:475)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6151//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6151//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6151//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6151//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6151//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6151//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6151//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6151//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6151//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6151//console
This message is automatically generated.
> Use Netflix's Curator zookeeper library
> ---
>
> Key: HBASE-7411
> URL: https://issues.apache.org/jira/browse/HBASE-7411
> Project: HBase
> Issue Type: New Feature
> Components: Zookeeper
>Affects Versions: 0.95.2
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.95.2
>
> Attachments: 7411v2.txt, 7411v2.txt, 7411v3.txt, 7411v4.txt,
> 7411v4.txt, hbase-7411_v0.patch
>
>
> We have mentioned using the Curator library
> (https://github.com/Netflix/curator) elsewhere but we can continue the
> discussion in this.
> The advantages for the curator lib over ours are the recipes. We have very
> similar retrying mechanism, and we don't need much of the nice client-API
> layer.
> We also have similar Listener interface, etc.
> I think we can decide on one of the following options:
> 1. Do not depend on curator. We have some of the recipes, and some custom
> recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991,
> etc). We can also copy / fork some code from there.
> 2. Replace all of our zk usage / connection management to curator. We may
> keep the current set of API's as a thin wrapper.
> 3. Use our own connection management / retry logic, and build a custom
> CuratorFramework implementation for the curator recipes. This will keep the
> current zk logic/code intact, and allow us to use curator-recipes as we see
> fit.
> 4. Allow both curator and our zk layer to manage the connection. We will
> still have 1 connection, but 2 abstraction layers sharing it. This is the
> easiest to implement, but a freak show?
> I have
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[
https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13694216#comment-13694216
]
Hadoop QA commented on HBASE-7411:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12589781/7411v4.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 9 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.access.TestAccessController
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.TestZooKeeper.testRegionAssignmentAfterMasterRecoveryDueToZKExpiry(TestZooKeeper.java:475)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6149//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6149//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6149//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6149//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6149//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6149//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6149//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6149//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6149//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6149//console
This message is automatically generated.
> Use Netflix's Curator zookeeper library
> ---
>
> Key: HBASE-7411
> URL: https://issues.apache.org/jira/browse/HBASE-7411
> Project: HBase
> Issue Type: New Feature
> Components: Zookeeper
>Affects Versions: 0.95.2
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.95.2
>
> Attachments: 7411v2.txt, 7411v2.txt, 7411v3.txt, 7411v4.txt,
> hbase-7411_v0.patch
>
>
> We have mentioned using the Curator library
> (https://github.com/Netflix/curator) elsewhere but we can continue the
> discussion in this.
> The advantages for the curator lib over ours are the recipes. We have very
> similar retrying mechanism, and we don't need much of the nice client-API
> layer.
> We also have similar Listener interface, etc.
> I think we can decide on one of the following options:
> 1. Do not depend on curator. We have some of the recipes, and some custom
> recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991,
> etc). We can also copy / fork some code from there.
> 2. Replace all of our zk usage / connection management to curator. We may
> keep the current set of API's as a thin wrapper.
> 3. Use our own connection management / retry logic, and build a custom
> CuratorFramework implementation for the curator recipes. This will keep the
> current zk logic/code intact, and allow us to use curator-recipes as we see
> fit.
> 4. Allow both curator and our zk layer to manage the connection. We will
> still have 1 connection, but 2 abstraction layers sharing it. This is the
> easiest to implement, but a freak show?
> I have a patch for 4, and now prototyping 2 or 3 whichever will be less
>
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[
https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13694143#comment-13694143
]
Hadoop QA commented on HBASE-7411:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12589780/7411v3.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 7 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6148//console
This message is automatically generated.
> Use Netflix's Curator zookeeper library
> ---
>
> Key: HBASE-7411
> URL: https://issues.apache.org/jira/browse/HBASE-7411
> Project: HBase
> Issue Type: New Feature
> Components: Zookeeper
>Affects Versions: 0.95.2
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.95.2
>
> Attachments: 7411v2.txt, 7411v2.txt, 7411v3.txt, hbase-7411_v0.patch
>
>
> We have mentioned using the Curator library
> (https://github.com/Netflix/curator) elsewhere but we can continue the
> discussion in this.
> The advantages for the curator lib over ours are the recipes. We have very
> similar retrying mechanism, and we don't need much of the nice client-API
> layer.
> We also have similar Listener interface, etc.
> I think we can decide on one of the following options:
> 1. Do not depend on curator. We have some of the recipes, and some custom
> recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991,
> etc). We can also copy / fork some code from there.
> 2. Replace all of our zk usage / connection management to curator. We may
> keep the current set of API's as a thin wrapper.
> 3. Use our own connection management / retry logic, and build a custom
> CuratorFramework implementation for the curator recipes. This will keep the
> current zk logic/code intact, and allow us to use curator-recipes as we see
> fit.
> 4. Allow both curator and our zk layer to manage the connection. We will
> still have 1 connection, but 2 abstraction layers sharing it. This is the
> easiest to implement, but a freak show?
> I have a patch for 4, and now prototyping 2 or 3 whichever will be less
> painful.
> Related issues:
> HBASE-5547
> HBASE-7305
> HBASE-7212
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[
https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13691558#comment-13691558
]
Hadoop QA commented on HBASE-7411:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12589289/7411v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestRegionServerNoMaster
org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort
org.apache.hadoop.hbase.master.TestMasterMetrics
org.apache.hadoop.hbase.client.TestHCM
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.zookeeper.TestRecoverableZooKeeper
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.rest.TestGzipFilter.testGzipFilter(TestGzipFilter.java:95)
at
org.apache.hadoop.hbase.TestZooKeeper.testRegionAssignmentAfterMasterRecoveryDueToZKExpiry(TestZooKeeper.java:475)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6109//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6109//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6109//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6109//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6109//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6109//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6109//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6109//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6109//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6109//console
This message is automatically generated.
> Use Netflix's Curator zookeeper library
> ---
>
> Key: HBASE-7411
> URL: https://issues.apache.org/jira/browse/HBASE-7411
> Project: HBase
> Issue Type: New Feature
> Components: Zookeeper
>Affects Versions: 0.95.2
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.95.2
>
> Attachments: 7411v2.txt, 7411v2.txt, hbase-7411_v0.patch
>
>
> We have mentioned using the Curator library
> (https://github.com/Netflix/curator) elsewhere but we can continue the
> discussion in this.
> The advantages for the curator lib over ours are the recipes. We have very
> similar retrying mechanism, and we don't need much of the nice client-API
> layer.
> We also have similar Listener interface, etc.
> I think we can decide on one of the following options:
> 1. Do not depend on curator. We have some of the recipes, and some custom
> recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991,
> etc). We can also copy / fork some code from there.
> 2. Replace all of our zk usage / connection management to curator. We may
> keep the current set of API's as a thin wrapper.
> 3. Use our own connection management / retry logic, and build a cust
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[
https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13691384#comment-13691384
]
Hadoop QA commented on HBASE-7411:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12589289/7411v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestMasterMetrics
org.apache.hadoop.hbase.zookeeper.TestRecoverableZooKeeper
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.client.TestHCM
org.apache.hadoop.hbase.regionserver.TestRegionServerNoMaster
org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.TestZooKeeper.testRegionAssignmentAfterMasterRecoveryDueToZKExpiry(TestZooKeeper.java:475)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6107//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6107//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6107//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6107//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6107//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6107//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6107//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6107//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6107//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6107//console
This message is automatically generated.
> Use Netflix's Curator zookeeper library
> ---
>
> Key: HBASE-7411
> URL: https://issues.apache.org/jira/browse/HBASE-7411
> Project: HBase
> Issue Type: New Feature
> Components: Zookeeper
>Affects Versions: 0.95.2
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.95.2
>
> Attachments: 7411v2.txt, 7411v2.txt, hbase-7411_v0.patch
>
>
> We have mentioned using the Curator library
> (https://github.com/Netflix/curator) elsewhere but we can continue the
> discussion in this.
> The advantages for the curator lib over ours are the recipes. We have very
> similar retrying mechanism, and we don't need much of the nice client-API
> layer.
> We also have similar Listener interface, etc.
> I think we can decide on one of the following options:
> 1. Do not depend on curator. We have some of the recipes, and some custom
> recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991,
> etc). We can also copy / fork some code from there.
> 2. Replace all of our zk usage / connection management to curator. We may
> keep the current set of API's as a thin wrapper.
> 3. Use our own connection management / retry logic, and build a custom
> CuratorFramework implementation for the curator recipes. This will keep the
> current zk logic/code intact,
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[
https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13691346#comment-13691346
]
Hadoop QA commented on HBASE-7411:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12589288/7411v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6101//console
This message is automatically generated.
> Use Netflix's Curator zookeeper library
> ---
>
> Key: HBASE-7411
> URL: https://issues.apache.org/jira/browse/HBASE-7411
> Project: HBase
> Issue Type: New Feature
> Components: Zookeeper
>Affects Versions: 0.95.2
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.95.2
>
> Attachments: 7411v2.txt, hbase-7411_v0.patch
>
>
> We have mentioned using the Curator library
> (https://github.com/Netflix/curator) elsewhere but we can continue the
> discussion in this.
> The advantages for the curator lib over ours are the recipes. We have very
> similar retrying mechanism, and we don't need much of the nice client-API
> layer.
> We also have similar Listener interface, etc.
> I think we can decide on one of the following options:
> 1. Do not depend on curator. We have some of the recipes, and some custom
> recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991,
> etc). We can also copy / fork some code from there.
> 2. Replace all of our zk usage / connection management to curator. We may
> keep the current set of API's as a thin wrapper.
> 3. Use our own connection management / retry logic, and build a custom
> CuratorFramework implementation for the curator recipes. This will keep the
> current zk logic/code intact, and allow us to use curator-recipes as we see
> fit.
> 4. Allow both curator and our zk layer to manage the connection. We will
> still have 1 connection, but 2 abstraction layers sharing it. This is the
> easiest to implement, but a freak show?
> I have a patch for 4, and now prototyping 2 or 3 whichever will be less
> painful.
> Related issues:
> HBASE-5547
> HBASE-7305
> HBASE-7212
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13654740#comment-13654740 ] Jordan Zimmerman commented on HBASE-7411: - The first Apache Curator release is out: http://mail-archives.apache.org/mod_mbox/incubator-curator-user/201305.mbox/%3CEDA26D30-171C-481F-B740-4284A1C3B417%40jordanzimmerman.com%3E > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.95.2 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-7411_v0.patch > > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13593715#comment-13593715 ] Jordan Zimmerman commented on HBASE-7411: - The vote is on right now - so here's your chance. > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-7411_v0.patch > > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13586591#comment-13586591 ] Jordan Zimmerman commented on HBASE-7411: - http://wiki.apache.org/incubator/CuratorProposal > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-7411_v0.patch > > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584604#comment-13584604 ] Andrew Purtell commented on HBASE-7411: --- bq. FYI - Curator will soon be going into Incubator. Stay tuned.. I'll hope for the best. The second that happens you can consider any objection I have here withdrawn. > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-7411_v0.patch > > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584601#comment-13584601 ] Enis Soztutar commented on HBASE-7411: -- [~randgalt] This is great news. Let me know if you need any help in Incubator. > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-7411_v0.patch > > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13583613#comment-13583613 ] Jordan Zimmerman commented on HBASE-7411: - FYI - Curator will soon be going into Incubator. Stay tuned... > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-7411_v0.patch > > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13574892#comment-13574892 ] Enis Soztutar commented on HBASE-7411: -- @Jordan, if you guys decide to bring curator to incubator, that will very well lower the boundary for our adoption. I can sign up if you need any help with the incubator process or ASF. > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-7411_v0.patch > > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13574878#comment-13574878 ] Jordan Zimmerman commented on HBASE-7411: - I'll bring it up with Netflix OSS folks and then continue with Patrick Hunt if needed. > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-7411_v0.patch > > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13574866#comment-13574866 ] Ted Yu commented on HBASE-7411: --- @Jordan: Do you mind continuing that discussion on zookeeper mailing list ? > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-7411_v0.patch > > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13574862#comment-13574862 ] Jordan Zimmerman commented on HBASE-7411: - Jordan Zimmerman (Curator Author here). I've been talking for a while with the ZK devs about putting Curator into Apache. The last discussions regarding this were last year. I'm willing to discuss again if needed. > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-7411_v0.patch > > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13554345#comment-13554345 ] Andrew Purtell commented on HBASE-7411: --- bq. Other issues w/ our current RZK Just to be clear I'm not arguing to keep RZK, or to ignore Curator code. > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-7411_v0.patch > > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13553516#comment-13553516 ] stack commented on HBASE-7411: -- Lets move discussion out to dev list. Other issues w/ our current RZK are 1., the 'noise' it writes at the start of the data block, the sequence id, a 'trick' the FB lads moved away from up in their 'redo' RecoveringZooKeeper (as opposed to our RecoverableZooKeeper), and 2., its not under active dev. > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-7411_v0.patch > > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13551540#comment-13551540 ] Andrew Purtell commented on HBASE-7411: --- I don't think depending on a third party single owner library for something so critical is a good idea. What assurances can be provided that when we trace a difficult to debug problem in prod to Curator we can get a contributed fix in a new Curator release in a timely manner? Won't we then be forking Curator with a custom patch and hostnig a curator-x.y-HBASE Maven artifact somewhere, just like we did with Junit/Surefire, which by the way is much more widely used. Importing proven code that we ourselves could change is no problem, otherwise I'd have to consider a veto. > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-7411_v0.patch > > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13551468#comment-13551468 ] Enis Soztutar commented on HBASE-7411: -- Thanks Ted for the review. I'll address those in the next one. >From my understanding so far for Curator, and the discussions here, we should >be doing either the approach in the patch, or close this as won't fix. I would >prefer to reach a consensus sooner, rather than later if possible. > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-7411_v0.patch > > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[
https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13547668#comment-13547668
]
Ted Yu commented on HBASE-7411:
---
Interesting patch.
{code}
+ * to recreate the connection. This class bridges bridges curator and our
zk-management.
{code}
'bridges' is repeated.
{code}
+WatcherSet doubleWatcher;
{code}
Since WatcherSet always contains two Watchers, should the class be called
DoubleWatcher ?
{code}
+public synchronized ZooKeeper newZooKeeper(String connectString, int
sessionTimeout, Watcher curatorWatcher,
{code}
Wrap long line.
{code}
+ if (LOG.isDebugEnabled()) {
+LOG.debug("Sending last zookeeper event to curator: " + lastEvent);
+ }
+ if (lastEvent != null) {
{code}
I think the log should be placed inside the if block (lastEvent != null)
{code}
+ static class ManagedZooKeeperFactory implements ZookeeperFactory {
{code}
The above class can be private, right ?
{code}
+ public void reconnect() throws IOException, InterruptedException {
+zkFactory.close(); //we don't want to close the CuratorClient
+connect();
{code}
reconnect() invokes two methods of ManagedZooKeeperFactory. Should reconnect()
delegate to a synchronized method, reconnect(), of ManagedZooKeeperFactory ?
{code}
+ public void close() throws InterruptedException {
+if (this.curatorClient != null) {
+ this.curatorClient.close();
+}
{code}
this.curatorClient should be set to null after the close() call.
> Use Netflix's Curator zookeeper library
> ---
>
> Key: HBASE-7411
> URL: https://issues.apache.org/jira/browse/HBASE-7411
> Project: HBase
> Issue Type: New Feature
> Components: Zookeeper
>Affects Versions: 0.96.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-7411_v0.patch
>
>
> We have mentioned using the Curator library
> (https://github.com/Netflix/curator) elsewhere but we can continue the
> discussion in this.
> The advantages for the curator lib over ours are the recipes. We have very
> similar retrying mechanism, and we don't need much of the nice client-API
> layer.
> We also have similar Listener interface, etc.
> I think we can decide on one of the following options:
> 1. Do not depend on curator. We have some of the recipes, and some custom
> recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991,
> etc). We can also copy / fork some code from there.
> 2. Replace all of our zk usage / connection management to curator. We may
> keep the current set of API's as a thin wrapper.
> 3. Use our own connection management / retry logic, and build a custom
> CuratorFramework implementation for the curator recipes. This will keep the
> current zk logic/code intact, and allow us to use curator-recipes as we see
> fit.
> 4. Allow both curator and our zk layer to manage the connection. We will
> still have 1 connection, but 2 abstraction layers sharing it. This is the
> easiest to implement, but a freak show?
> I have a patch for 4, and now prototyping 2 or 3 whichever will be less
> painful.
> Related issues:
> HBASE-5547
> HBASE-7305
> HBASE-7212
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13537701#comment-13537701 ] Andrew Purtell commented on HBASE-7411: --- I get it. Using the recipes ... importing proven code would be ok. Adding more and more Maven dependencies to small single owner projects is the concern. If we think that we can/should fork at some future time if we need to bugfix one or more of them in 6 months or 2 years and the owner has gone away or in a different direction, then ok. I have no idea if we'd end up in that position with Curator of course. But ZK is so critical to HBase. > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13537691#comment-13537691 ] stack commented on HBASE-7411: -- Yeah our zk hookup is critical but it doesn't get much by way of attention given its import (and yes, we need to do some dependency pruning) It looks like dev on our 'curator', our custom zk client, is in a dead end and may be abandoned going by whats is over in the fb java commons. RecoverableZooKeeper became RecoveringZooKeeper and it doesn't do stuff like write serialization id as first few bytes of the znode data as ours does: https://github.com/facebook/jcommon/blob/master/zookeeper/src/main/java/com/facebook/zookeeper/RecoveringZooKeeper.java (We can ask the fb lads for the definitive). Our zk interface is also currently a mess w/ code spread all about the place w/ a strange layering that needs undoing (zookeeper package, ZKUtil, ZKAssign... clients import them all... then there is callback handling distributed throughout). We could with do a revisit hereabouts in general. Curator is used in more than just one project so has probably some extra resilience built up. It gets good reviews from zk fellas we all know and is under active development last time I checked. It does basic stuff like keeping the actual zookeeper instance from you so session can be reestablished behind the scenes (we have actual zookeeper instance everywhere and are hosed once session is gone... could entertain retries w/ curator as is, unless we did more code on our part). Implementing the reciepes is more new code in hbase w/ attendant bug fixing but if they are already done in another open source project, after some vetting and test, why not use it instead (as is, we need distributed locks, likely the double blocking receipe ). Just saying. > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13537621#comment-13537621 ] Andrew Purtell commented on HBASE-7411: --- I would not be in favor of direct use of the Curator library (as a Maven dependency). > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13537580#comment-13537580 ] Enis Soztutar commented on HBASE-7411: -- bq. Curator – HBase's use of ZooKeeper is absolutely critical. This should not be outsourced, especially outside of an ASF Hadoop-ish project Agreed that it is super critical that our zk usage is stable, debuggable, and fixable. Again, the main reason for this issue is to reuse their recipes (especially locks, and maybe queues, barriers and shared values). That is why I am relying more on 3. If we find that the recipes are useful on their own, we don't need to reimplement them, if not, we might as well just continue to custom build our recipes. > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
[ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13537482#comment-13537482 ] Andrew Purtell commented on HBASE-7411: --- I have the same concern here as was expressed on another JIRA recently regards htrace. It may not be a good idea to depend on a single developer project i.e. not an ASF community project. There are several instances I can think of now off the top of my head, ordered from least to greatest concern: - Cliff Click's high-scale-lib -- A single use case and a very stable library, not much of a concern - Yammer metrics -- Straightforward use of a stable library, and can be replaced without too much difficulty - htrace -- Without being an ASF Hadoop-ish project will we see coverage in HDFS and without that would adding spans in HBase code deliver enough of a return? I.e., unsure about the longer term value proposition. - Curator -- HBase's use of ZooKeeper is absolutely critical. This should not be outsourced, especially outside of an ASF Hadoop-ish project. > Use Netflix's Curator zookeeper library > --- > > Key: HBASE-7411 > URL: https://issues.apache.org/jira/browse/HBASE-7411 > Project: HBase > Issue Type: New Feature > Components: Zookeeper >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > > We have mentioned using the Curator library > (https://github.com/Netflix/curator) elsewhere but we can continue the > discussion in this. > The advantages for the curator lib over ours are the recipes. We have very > similar retrying mechanism, and we don't need much of the nice client-API > layer. > We also have similar Listener interface, etc. > I think we can decide on one of the following options: > 1. Do not depend on curator. We have some of the recipes, and some custom > recipes (ZKAssign, Leader election, etc already working, locks in HBASE-5991, > etc). We can also copy / fork some code from there. > 2. Replace all of our zk usage / connection management to curator. We may > keep the current set of API's as a thin wrapper. > 3. Use our own connection management / retry logic, and build a custom > CuratorFramework implementation for the curator recipes. This will keep the > current zk logic/code intact, and allow us to use curator-recipes as we see > fit. > 4. Allow both curator and our zk layer to manage the connection. We will > still have 1 connection, but 2 abstraction layers sharing it. This is the > easiest to implement, but a freak show? > I have a patch for 4, and now prototyping 2 or 3 whichever will be less > painful. > Related issues: > HBASE-5547 > HBASE-7305 > HBASE-7212 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
