Re: DISCUSSION: 1.0.0
Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
Hi Aditya, I think it would be good to have a native client API in 1.0. I can look at the patches when back in the office later this week. On Jul 7, 2014, at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
Would you mind including the JIRA numbers along with the request? Thanks, Nick On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
The following is another candidate for 1.0.0 release HBASE-11447 Proposal for a generic transaction API for HBase Cheers On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
I thought the scope of HBASE-11447 is to review and refine proposed design, not to track the implementation of the feature? -Mikhail 2014-07-07 10:40 GMT-07:00 Andrew Purtell andrew.purt...@gmail.com: That is not a realistic proposal for 1.0 as far as I can see. There is only a tentative design spec on that issue. I assume we are sticking with Enis' earlier stated intent to get 1.0 out soon. On Jul 7, 2014, at 10:30 AM, Ted Yu yuzhih...@gmail.com wrote: The following is another candidate for 1.0.0 release HBASE-11447 Proposal for a generic transaction API for HBase Cheers On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
I think it is fair to say for both C API and transaction API that it should be ok to include them if they are ready before the time to cut the 1.0 RC. If not, I do not want to hold the release waiting for those. Enis On Mon, Jul 7, 2014 at 10:50 AM, Andrew Purtell andrew.purt...@gmail.com wrote: And by definition of that being a proposed design, there is no implementation to track (yet). There being no implementation, it's hard to see how it makes a 1.0 happening soon. Don't misunderstand me to be opposed to the idea or feature, far from it. I don't think Ted did anyone a service bringing it up in the context of 1.0. Already we are on some kind of detour from discussing things where there are patches available. On Jul 7, 2014, at 10:43 AM, Mikhail Antonov olorinb...@gmail.com wrote: I thought the scope of HBASE-11447 is to review and refine proposed design, not to track the implementation of the feature? -Mikhail 2014-07-07 10:40 GMT-07:00 Andrew Purtell andrew.purt...@gmail.com: That is not a realistic proposal for 1.0 as far as I can see. There is only a tentative design spec on that issue. I assume we are sticking with Enis' earlier stated intent to get 1.0 out soon. On Jul 7, 2014, at 10:30 AM, Ted Yu yuzhih...@gmail.com wrote: The following is another candidate for 1.0.0 release HBASE-11447 Proposal for a generic transaction API for HBase Cheers On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
And by definition of that being a proposed design, there is no implementation to track (yet). There being no implementation, it's hard to see how it makes a 1.0 happening soon. Don't misunderstand me to be opposed to the idea or feature, far from it. I don't think Ted did anyone a service bringing it up in the context of 1.0. Already we are on some kind of detour from discussing things where there are patches available. On Jul 7, 2014, at 10:43 AM, Mikhail Antonov olorinb...@gmail.com wrote: I thought the scope of HBASE-11447 is to review and refine proposed design, not to track the implementation of the feature? -Mikhail 2014-07-07 10:40 GMT-07:00 Andrew Purtell andrew.purt...@gmail.com: That is not a realistic proposal for 1.0 as far as I can see. There is only a tentative design spec on that issue. I assume we are sticking with Enis' earlier stated intent to get 1.0 out soon. On Jul 7, 2014, at 10:30 AM, Ted Yu yuzhih...@gmail.com wrote: The following is another candidate for 1.0.0 release HBASE-11447 Proposal for a generic transaction API for HBase Cheers On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov -- Thanks, Michael Antonov
[jira] [Created] (HBASE-11466) HConnectionImplementation should not use ZK
Mikhail Antonov created HBASE-11466: --- Summary: HConnectionImplementation should not use ZK Key: HBASE-11466 URL: https://issues.apache.org/jira/browse/HBASE-11466 Project: HBase Issue Type: Sub-task Components: Client Affects Versions: 2.0.0 Reporter: Mikhail Antonov Fix For: 2.0.0 Currently ConnectionManager.HConnectionImplementation uses ZK to get address of current master. Should instead use pluggable interface to get location of master to connect to (current active master in the cluster, until we have multiple active masters) elsewhere (e.g. round-robin over the list of masters set in the client's hbase-site.xml) -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: DISCUSSION: 1.0.0
Sorry about that. Here is the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 On Mon, Jul 7, 2014 at 10:05 AM, Nick Dimiduk ndimi...@gmail.com wrote: Would you mind including the JIRA numbers along with the request? Thanks, Nick On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov
[jira] [Created] (HBASE-11467) New impl of Registry interface not using ZK + new RPCs on master protocol.
Mikhail Antonov created HBASE-11467: --- Summary: New impl of Registry interface not using ZK + new RPCs on master protocol. Key: HBASE-11467 URL: https://issues.apache.org/jira/browse/HBASE-11467 Project: HBase Issue Type: Sub-task Reporter: Mikhail Antonov Currently there' only one implementation of Registry interface, which is using ZK to get info about meta. Need to create implementation which will be using RPC calls to master the client is connected to. That includes adding several new methods to master RPC protocol: - GetMetaRegionLocation - GetClusterId - IsTableOnlineState - GetCurrentNrHRS -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11468) MasterAddressTracker needs to be moved to hbase-server
Mikhail Antonov created HBASE-11468: --- Summary: MasterAddressTracker needs to be moved to hbase-server Key: HBASE-11468 URL: https://issues.apache.org/jira/browse/HBASE-11468 Project: HBase Issue Type: Sub-task Reporter: Mikhail Antonov Later we should get rid of it. For now, for the purpose to abstract client from ZK, we don't use in in hbase-client (where we read master location elsewhere), but on the server side we can use it for now, and still keep current znode tracking location of current active master. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11469) MetaTableLocator should be moved to hbase-server
Mikhail Antonov created HBASE-11469: --- Summary: MetaTableLocator should be moved to hbase-server Key: HBASE-11469 URL: https://issues.apache.org/jira/browse/HBASE-11469 Project: HBase Issue Type: Sub-task Reporter: Mikhail Antonov It shall not be used on client side, clients shall make rpc calls to master to find out about meta. Also it needs to either be removed from Server interface available on client side and moved into RegionServer class or something, or whole Server interface needs to be moved to hbase-server. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11470) Move Server interface to hbase-server
Mikhail Antonov created HBASE-11470: --- Summary: Move Server interface to hbase-server Key: HBASE-11470 URL: https://issues.apache.org/jira/browse/HBASE-11470 Project: HBase Issue Type: Sub-task Reporter: Mikhail Antonov Looks like it's not being used on client side anyway. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: DISCUSSION: 1.0.0
We're on the same page, Enis. Cheers On Mon, Jul 7, 2014 at 10:56 AM, Enis Söztutar e...@apache.org wrote: I think it is fair to say for both C API and transaction API that it should be ok to include them if they are ready before the time to cut the 1.0 RC. If not, I do not want to hold the release waiting for those. Enis On Mon, Jul 7, 2014 at 10:50 AM, Andrew Purtell andrew.purt...@gmail.com wrote: And by definition of that being a proposed design, there is no implementation to track (yet). There being no implementation, it's hard to see how it makes a 1.0 happening soon. Don't misunderstand me to be opposed to the idea or feature, far from it. I don't think Ted did anyone a service bringing it up in the context of 1.0. Already we are on some kind of detour from discussing things where there are patches available. On Jul 7, 2014, at 10:43 AM, Mikhail Antonov olorinb...@gmail.com wrote: I thought the scope of HBASE-11447 is to review and refine proposed design, not to track the implementation of the feature? -Mikhail 2014-07-07 10:40 GMT-07:00 Andrew Purtell andrew.purt...@gmail.com: That is not a realistic proposal for 1.0 as far as I can see. There is only a tentative design spec on that issue. I assume we are sticking with Enis' earlier stated intent to get 1.0 out soon. On Jul 7, 2014, at 10:30 AM, Ted Yu yuzhih...@gmail.com wrote: The following is another candidate for 1.0.0 release HBASE-11447 Proposal for a generic transaction API for HBase Cheers On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov -- Thanks, Michael Antonov
[jira] [Created] (HBASE-11471) Move TableStateManager and ZkTableStateManager to hbase-server
Mikhail Antonov created HBASE-11471: --- Summary: Move TableStateManager and ZkTableStateManager to hbase-server Key: HBASE-11471 URL: https://issues.apache.org/jira/browse/HBASE-11471 Project: HBase Issue Type: Sub-task Reporter: Mikhail Antonov -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11472) Remove ZKTableStateClientSideReader class
Mikhail Antonov created HBASE-11472: --- Summary: Remove ZKTableStateClientSideReader class Key: HBASE-11472 URL: https://issues.apache.org/jira/browse/HBASE-11472 Project: HBase Issue Type: Sub-task Reporter: Mikhail Antonov On client side this is now only used by ZooKeeperRegistry. Shouldn't be needed once we have registry implementation not using ZK. On server side we should be using TableStateManager instead. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11473) Add BaseWALObserver class
Enis Soztutar created HBASE-11473: - Summary: Add BaseWALObserver class Key: HBASE-11473 URL: https://issues.apache.org/jira/browse/HBASE-11473 Project: HBase Issue Type: Improvement Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.99.0, 0.98.4, 2.0.0 We lack a BaseWALObserver class for WALObserver coprocessors. Other coprocessor interfaces come with a default base class which makes life a little bit easier for coprocessor implementors. I think we can add this to 0.98+. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: DISCUSSION: 1.0.0
I agree. These are fine things we could get in 1.1 or 1.2 release if they don't break compat and otherwise a 2.0 release timeframe. Jon. On Mon, Jul 7, 2014 at 10:56 AM, Enis Söztutar e...@apache.org wrote: I think it is fair to say for both C API and transaction API that it should be ok to include them if they are ready before the time to cut the 1.0 RC. If not, I do not want to hold the release waiting for those. Enis On Mon, Jul 7, 2014 at 10:50 AM, Andrew Purtell andrew.purt...@gmail.com wrote: And by definition of that being a proposed design, there is no implementation to track (yet). There being no implementation, it's hard to see how it makes a 1.0 happening soon. Don't misunderstand me to be opposed to the idea or feature, far from it. I don't think Ted did anyone a service bringing it up in the context of 1.0. Already we are on some kind of detour from discussing things where there are patches available. On Jul 7, 2014, at 10:43 AM, Mikhail Antonov olorinb...@gmail.com wrote: I thought the scope of HBASE-11447 is to review and refine proposed design, not to track the implementation of the feature? -Mikhail 2014-07-07 10:40 GMT-07:00 Andrew Purtell andrew.purt...@gmail.com: That is not a realistic proposal for 1.0 as far as I can see. There is only a tentative design spec on that issue. I assume we are sticking with Enis' earlier stated intent to get 1.0 out soon. On Jul 7, 2014, at 10:30 AM, Ted Yu yuzhih...@gmail.com wrote: The following is another candidate for 1.0.0 release HBASE-11447 Proposal for a generic transaction API for HBase Cheers On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov -- Thanks, Michael Antonov -- // Jonathan Hsieh (shay) // HBase Tech Lead, Software Engineer, Cloudera // j...@cloudera.com // @jmhsieh
[jira] [Created] (HBASE-11474) [Thrift2] support authentication/impersonation
Jimmy Xiang created HBASE-11474: --- Summary: [Thrift2] support authentication/impersonation Key: HBASE-11474 URL: https://issues.apache.org/jira/browse/HBASE-11474 Project: HBase Issue Type: Improvement Reporter: Jimmy Xiang Assignee: Jimmy Xiang This is to do the same as HBASE-11349 for Thrift2. -- This message was sent by Atlassian JIRA (v6.2#6252)
Jenkins Build Slaves
All Yahoo is in the process of retiring all the hadoop jenkins build slaves, *hadoop[1-9]* and replace them with a newer set of beefier hosts. These new machines are configured with *ubuntu-14.04*. Over the next couple of days I will be configuring the build jobs to run on these newly configured build slaves. To automate the installation of tools and build libraries I have put together ansible scripts and here is the link to the toolchain repo. *https://github.com/apache/toolchain https://github.com/apache/toolchain* During the transition, the old build slave will be accessible, and expected to be shutdown by 07/15. I will send out an update later this week when this transition is complete. *Mean while, I would like to request the project owners to remove/cleanup any stale * *jenkins job for their respective project and help with any builds issue to make this * *transition seamless. * Thanks - Giri -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Created] (HBASE-11475) Distributed log replay should also replay compaction events
Enis Soztutar created HBASE-11475: - Summary: Distributed log replay should also replay compaction events Key: HBASE-11475 URL: https://issues.apache.org/jira/browse/HBASE-11475 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.99.0, 0.98.4, 2.0.0 Compaction events are written to WAL as of HBASE-2231 got committed. However, it seems that distributed log replay skips those entries without sending it to the replaying region. In contrast log split replays those. It seems that we should fix log replay to also replay the compaction events. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Reopened] (HBASE-11293) Master and Region servers fail to start when hbase.master.ipc.address=0.0.0.0, hbase.regionserver.ipc.address=0.0.0.0 and Kerberos is enabled
[ https://issues.apache.org/jira/browse/HBASE-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das reopened HBASE-11293: - [~miharp], this is not done yet. Trunk needs to have a patch. Master and Region servers fail to start when hbase.master.ipc.address=0.0.0.0, hbase.regionserver.ipc.address=0.0.0.0 and Kerberos is enabled - Key: HBASE-11293 URL: https://issues.apache.org/jira/browse/HBASE-11293 Project: HBase Issue Type: Bug Reporter: Michael Harp Assignee: Devaraj Das Attachments: 11293-1.txt Setting {code} hbase.master.ipc.address=0.0.0.0 hbase.regionserver.ipc.address=0.0.0.0 {code} causes the _HOST substitution in hbase/_h...@example.com to result in hbase/0:0:0:0:0:0:0:0...@example.com which in turn causes kerberos authentication to fail. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11476) Expand 'Conceptual View' section of Data Model chapter
Misty Stanley-Jones created HBASE-11476: --- Summary: Expand 'Conceptual View' section of Data Model chapter Key: HBASE-11476 URL: https://issues.apache.org/jira/browse/HBASE-11476 Project: HBase Issue Type: Bug Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Could use some updating and expansion to emphasize the differences between HBase and an RDBMS. I found http://jimbojw.com/wiki/index.php?title=Understanding_Hbase_and_BigTable which is just excellent and we should link to it. -- This message was sent by Atlassian JIRA (v6.2#6252)