Re: DISCUSSION: 1.0.0
Hey, Yeah, was busy with something else (HBASE-10070 subtasks) for the last couple of weeks. I intend to get back to 0.99 real soon. Any help would be awesome. I'll call out for an RC next week. Enis On Fri, Aug 22, 2014 at 1:47 PM, Stack st...@duboce.net wrote: How we looking for a 0.99.0? I can go review of outstanding issue list np Enis, just say, but you probably have a notion on where we are already. Grand, St.Ack On Tue, Jul 29, 2014 at 2:35 AM, Aditya adityakish...@gmail.com wrote: Thanks Ted! On Mon, Jul 28, 2014 at 4:37 PM, Ted Yu yuzhih...@gmail.com wrote: I started with https://reviews.apache.org/r/23175/ Will continue reviewing this week. On Mon, Jul 28, 2014 at 1:40 PM, Aditya adityakish...@gmail.com wrote: Did anyone get a chance to take a look at the patches? Regards, aditya... On Sun, Jul 20, 2014 at 8:37 PM, Aditya adityakish...@gmail.com wrote: The wrapper jar is part of the first patch, which is in git mailbox patch format. On Sat, Jul 19, 2014 at 2:03 AM, Ted Yu yuzhih...@gmail.com wrote: You may want to attach the wrapper jar to the JIRA directly. Cheers On Jul 19, 2014, at 1:52 AM, Aditya adityakish...@gmail.com wrote: Looks like the regular patch command skips any binary included in patches. On Sat, Jul 19, 2014 at 1:37 AM, Aditya adityakish...@gmail.com wrote: Thanks for taking a look Ted! Looks like the second patch created with git diff excluded the Gradle wrapper JAR from the patch. I would generate a new one which includes this this jar. In the meantime, you should be able to use the first patch attached to the JIRA which is in git-am format and that would let you build. On Fri, Jul 18, 2014 at 3:40 PM, Ted Yu yuzhih...@gmail.com wrote: Nice work, Aditya. Looks like the hbase-native-client profile requires gradle ? [exec] Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain Will take a look at your patch. Cheers On Fri, Jul 18, 2014 at 3:08 PM, Aditya adityakish...@gmail.com wrote: As requested, I have attached a combined patch to the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 and submitted it to jenkins. Would be great if someone could take a look and provide feedback. Regards, aditya... On Wed, Jul 9, 2014 at 7:05 PM, Aditya adityakish...@gmail.com wrote: I was hoping to get some initial comments before attaching patches for the build boat. I have broken the entire code into 5 patch sets, layered in a sequnce, each focusing on a particular area (public headers/JNI implementation/Examples+unit test, etc) for the ease of review. https://reviews.apache.org/r/23175/ https://reviews.apache.org/r/23176/ https://reviews.apache.org/r/23177/ https://reviews.apache.org/r/23178/ https://reviews.apache.org/r/23179/ These are also available as a sequence of patches as the pull request https://github.com/apache/hbase/pull/1. Only the last patch hooks everything to the HBase build process (optionally) and hence I was thinking of squashing these separate patches into a single patch to be submitted for build. On Wed, Jul 9, 2014 at 11:32 AM, Nick Dimiduk ndimi...@gmail.com wrote: This ticket has only open subtasks, ie nothing in 'patch available'. I assume you mean HBASE-10168. We'll see about getting you some reviews, but you should also go about formatting the patch for buildbot. Also, since your 3 reviews are individually 100+k, you should consider breaking them into three separate tickets. my 2¢ -n On Mon, Jul 7, 2014 at 12:01 PM, Aditya adityakish...@gmail.com wrote: Sorry about that. Here is the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 On Mon, Jul 7, 2014 at 10:05 AM, Nick Dimiduk ndimi...@gmail.com wrote: Would you mind including the JIRA numbers along with the request? Thanks, Nick On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1?
Re: DISCUSSION: 1.0.0
Thanks Ted! On Mon, Jul 28, 2014 at 4:37 PM, Ted Yu yuzhih...@gmail.com wrote: I started with https://reviews.apache.org/r/23175/ Will continue reviewing this week. On Mon, Jul 28, 2014 at 1:40 PM, Aditya adityakish...@gmail.com wrote: Did anyone get a chance to take a look at the patches? Regards, aditya... On Sun, Jul 20, 2014 at 8:37 PM, Aditya adityakish...@gmail.com wrote: The wrapper jar is part of the first patch, which is in git mailbox patch format. On Sat, Jul 19, 2014 at 2:03 AM, Ted Yu yuzhih...@gmail.com wrote: You may want to attach the wrapper jar to the JIRA directly. Cheers On Jul 19, 2014, at 1:52 AM, Aditya adityakish...@gmail.com wrote: Looks like the regular patch command skips any binary included in patches. On Sat, Jul 19, 2014 at 1:37 AM, Aditya adityakish...@gmail.com wrote: Thanks for taking a look Ted! Looks like the second patch created with git diff excluded the Gradle wrapper JAR from the patch. I would generate a new one which includes this this jar. In the meantime, you should be able to use the first patch attached to the JIRA which is in git-am format and that would let you build. On Fri, Jul 18, 2014 at 3:40 PM, Ted Yu yuzhih...@gmail.com wrote: Nice work, Aditya. Looks like the hbase-native-client profile requires gradle ? [exec] Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain Will take a look at your patch. Cheers On Fri, Jul 18, 2014 at 3:08 PM, Aditya adityakish...@gmail.com wrote: As requested, I have attached a combined patch to the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 and submitted it to jenkins. Would be great if someone could take a look and provide feedback. Regards, aditya... On Wed, Jul 9, 2014 at 7:05 PM, Aditya adityakish...@gmail.com wrote: I was hoping to get some initial comments before attaching patches for the build boat. I have broken the entire code into 5 patch sets, layered in a sequnce, each focusing on a particular area (public headers/JNI implementation/Examples+unit test, etc) for the ease of review. https://reviews.apache.org/r/23175/ https://reviews.apache.org/r/23176/ https://reviews.apache.org/r/23177/ https://reviews.apache.org/r/23178/ https://reviews.apache.org/r/23179/ These are also available as a sequence of patches as the pull request https://github.com/apache/hbase/pull/1. Only the last patch hooks everything to the HBase build process (optionally) and hence I was thinking of squashing these separate patches into a single patch to be submitted for build. On Wed, Jul 9, 2014 at 11:32 AM, Nick Dimiduk ndimi...@gmail.com wrote: This ticket has only open subtasks, ie nothing in 'patch available'. I assume you mean HBASE-10168. We'll see about getting you some reviews, but you should also go about formatting the patch for buildbot. Also, since your 3 reviews are individually 100+k, you should consider breaking them into three separate tickets. my 2¢ -n On Mon, Jul 7, 2014 at 12:01 PM, Aditya adityakish...@gmail.com wrote: Sorry about that. Here is the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 On Mon, Jul 7, 2014 at 10:05 AM, Nick Dimiduk ndimi...@gmail.com wrote: Would you mind including the JIRA numbers along with the request? Thanks, Nick On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log
Re: DISCUSSION: 1.0.0
Did anyone get a chance to take a look at the patches? Regards, aditya... On Sun, Jul 20, 2014 at 8:37 PM, Aditya adityakish...@gmail.com wrote: The wrapper jar is part of the first patch, which is in git mailbox patch format. On Sat, Jul 19, 2014 at 2:03 AM, Ted Yu yuzhih...@gmail.com wrote: You may want to attach the wrapper jar to the JIRA directly. Cheers On Jul 19, 2014, at 1:52 AM, Aditya adityakish...@gmail.com wrote: Looks like the regular patch command skips any binary included in patches. On Sat, Jul 19, 2014 at 1:37 AM, Aditya adityakish...@gmail.com wrote: Thanks for taking a look Ted! Looks like the second patch created with git diff excluded the Gradle wrapper JAR from the patch. I would generate a new one which includes this this jar. In the meantime, you should be able to use the first patch attached to the JIRA which is in git-am format and that would let you build. On Fri, Jul 18, 2014 at 3:40 PM, Ted Yu yuzhih...@gmail.com wrote: Nice work, Aditya. Looks like the hbase-native-client profile requires gradle ? [exec] Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain Will take a look at your patch. Cheers On Fri, Jul 18, 2014 at 3:08 PM, Aditya adityakish...@gmail.com wrote: As requested, I have attached a combined patch to the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 and submitted it to jenkins. Would be great if someone could take a look and provide feedback. Regards, aditya... On Wed, Jul 9, 2014 at 7:05 PM, Aditya adityakish...@gmail.com wrote: I was hoping to get some initial comments before attaching patches for the build boat. I have broken the entire code into 5 patch sets, layered in a sequnce, each focusing on a particular area (public headers/JNI implementation/Examples+unit test, etc) for the ease of review. https://reviews.apache.org/r/23175/ https://reviews.apache.org/r/23176/ https://reviews.apache.org/r/23177/ https://reviews.apache.org/r/23178/ https://reviews.apache.org/r/23179/ These are also available as a sequence of patches as the pull request https://github.com/apache/hbase/pull/1. Only the last patch hooks everything to the HBase build process (optionally) and hence I was thinking of squashing these separate patches into a single patch to be submitted for build. On Wed, Jul 9, 2014 at 11:32 AM, Nick Dimiduk ndimi...@gmail.com wrote: This ticket has only open subtasks, ie nothing in 'patch available'. I assume you mean HBASE-10168. We'll see about getting you some reviews, but you should also go about formatting the patch for buildbot. Also, since your 3 reviews are individually 100+k, you should consider breaking them into three separate tickets. my 2¢ -n On Mon, Jul 7, 2014 at 12:01 PM, Aditya adityakish...@gmail.com wrote: Sorry about that. Here is the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 On Mon, Jul 7, 2014 at 10:05 AM, Nick Dimiduk ndimi...@gmail.com wrote: Would you mind including the JIRA numbers along with the request? Thanks, Nick On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction
Re: DISCUSSION: 1.0.0
I started with https://reviews.apache.org/r/23175/ Will continue reviewing this week. On Mon, Jul 28, 2014 at 1:40 PM, Aditya adityakish...@gmail.com wrote: Did anyone get a chance to take a look at the patches? Regards, aditya... On Sun, Jul 20, 2014 at 8:37 PM, Aditya adityakish...@gmail.com wrote: The wrapper jar is part of the first patch, which is in git mailbox patch format. On Sat, Jul 19, 2014 at 2:03 AM, Ted Yu yuzhih...@gmail.com wrote: You may want to attach the wrapper jar to the JIRA directly. Cheers On Jul 19, 2014, at 1:52 AM, Aditya adityakish...@gmail.com wrote: Looks like the regular patch command skips any binary included in patches. On Sat, Jul 19, 2014 at 1:37 AM, Aditya adityakish...@gmail.com wrote: Thanks for taking a look Ted! Looks like the second patch created with git diff excluded the Gradle wrapper JAR from the patch. I would generate a new one which includes this this jar. In the meantime, you should be able to use the first patch attached to the JIRA which is in git-am format and that would let you build. On Fri, Jul 18, 2014 at 3:40 PM, Ted Yu yuzhih...@gmail.com wrote: Nice work, Aditya. Looks like the hbase-native-client profile requires gradle ? [exec] Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain Will take a look at your patch. Cheers On Fri, Jul 18, 2014 at 3:08 PM, Aditya adityakish...@gmail.com wrote: As requested, I have attached a combined patch to the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 and submitted it to jenkins. Would be great if someone could take a look and provide feedback. Regards, aditya... On Wed, Jul 9, 2014 at 7:05 PM, Aditya adityakish...@gmail.com wrote: I was hoping to get some initial comments before attaching patches for the build boat. I have broken the entire code into 5 patch sets, layered in a sequnce, each focusing on a particular area (public headers/JNI implementation/Examples+unit test, etc) for the ease of review. https://reviews.apache.org/r/23175/ https://reviews.apache.org/r/23176/ https://reviews.apache.org/r/23177/ https://reviews.apache.org/r/23178/ https://reviews.apache.org/r/23179/ These are also available as a sequence of patches as the pull request https://github.com/apache/hbase/pull/1. Only the last patch hooks everything to the HBase build process (optionally) and hence I was thinking of squashing these separate patches into a single patch to be submitted for build. On Wed, Jul 9, 2014 at 11:32 AM, Nick Dimiduk ndimi...@gmail.com wrote: This ticket has only open subtasks, ie nothing in 'patch available'. I assume you mean HBASE-10168. We'll see about getting you some reviews, but you should also go about formatting the patch for buildbot. Also, since your 3 reviews are individually 100+k, you should consider breaking them into three separate tickets. my 2¢ -n On Mon, Jul 7, 2014 at 12:01 PM, Aditya adityakish...@gmail.com wrote: Sorry about that. Here is the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 On Mon, Jul 7, 2014 at 10:05 AM, Nick Dimiduk ndimi...@gmail.com wrote: Would you mind including the JIRA numbers along with the request? Thanks, Nick On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is
Re: DISCUSSION: 1.0.0
The wrapper jar is part of the first patch, which is in git mailbox patch format. On Sat, Jul 19, 2014 at 2:03 AM, Ted Yu yuzhih...@gmail.com wrote: You may want to attach the wrapper jar to the JIRA directly. Cheers On Jul 19, 2014, at 1:52 AM, Aditya adityakish...@gmail.com wrote: Looks like the regular patch command skips any binary included in patches. On Sat, Jul 19, 2014 at 1:37 AM, Aditya adityakish...@gmail.com wrote: Thanks for taking a look Ted! Looks like the second patch created with git diff excluded the Gradle wrapper JAR from the patch. I would generate a new one which includes this this jar. In the meantime, you should be able to use the first patch attached to the JIRA which is in git-am format and that would let you build. On Fri, Jul 18, 2014 at 3:40 PM, Ted Yu yuzhih...@gmail.com wrote: Nice work, Aditya. Looks like the hbase-native-client profile requires gradle ? [exec] Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain Will take a look at your patch. Cheers On Fri, Jul 18, 2014 at 3:08 PM, Aditya adityakish...@gmail.com wrote: As requested, I have attached a combined patch to the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 and submitted it to jenkins. Would be great if someone could take a look and provide feedback. Regards, aditya... On Wed, Jul 9, 2014 at 7:05 PM, Aditya adityakish...@gmail.com wrote: I was hoping to get some initial comments before attaching patches for the build boat. I have broken the entire code into 5 patch sets, layered in a sequnce, each focusing on a particular area (public headers/JNI implementation/Examples+unit test, etc) for the ease of review. https://reviews.apache.org/r/23175/ https://reviews.apache.org/r/23176/ https://reviews.apache.org/r/23177/ https://reviews.apache.org/r/23178/ https://reviews.apache.org/r/23179/ These are also available as a sequence of patches as the pull request https://github.com/apache/hbase/pull/1. Only the last patch hooks everything to the HBase build process (optionally) and hence I was thinking of squashing these separate patches into a single patch to be submitted for build. On Wed, Jul 9, 2014 at 11:32 AM, Nick Dimiduk ndimi...@gmail.com wrote: This ticket has only open subtasks, ie nothing in 'patch available'. I assume you mean HBASE-10168. We'll see about getting you some reviews, but you should also go about formatting the patch for buildbot. Also, since your 3 reviews are individually 100+k, you should consider breaking them into three separate tickets. my 2¢ -n On Mon, Jul 7, 2014 at 12:01 PM, Aditya adityakish...@gmail.com wrote: Sorry about that. Here is the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 On Mon, Jul 7, 2014 at 10:05 AM, Nick Dimiduk ndimi...@gmail.com wrote: Would you mind including the JIRA numbers along with the request? Thanks, Nick On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to
Re: DISCUSSION: 1.0.0
Thanks for taking a look Ted! Looks like the second patch created with git diff excluded the Gradle wrapper JAR from the patch. I would generate a new one which includes this this jar. In the meantime, you should be able to use the first patch attached to the JIRA which is in git-am format and that would let you build. On Fri, Jul 18, 2014 at 3:40 PM, Ted Yu yuzhih...@gmail.com wrote: Nice work, Aditya. Looks like the hbase-native-client profile requires gradle ? [exec] Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain Will take a look at your patch. Cheers On Fri, Jul 18, 2014 at 3:08 PM, Aditya adityakish...@gmail.com wrote: As requested, I have attached a combined patch to the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 and submitted it to jenkins. Would be great if someone could take a look and provide feedback. Regards, aditya... On Wed, Jul 9, 2014 at 7:05 PM, Aditya adityakish...@gmail.com wrote: I was hoping to get some initial comments before attaching patches for the build boat. I have broken the entire code into 5 patch sets, layered in a sequnce, each focusing on a particular area (public headers/JNI implementation/Examples+unit test, etc) for the ease of review. https://reviews.apache.org/r/23175/ https://reviews.apache.org/r/23176/ https://reviews.apache.org/r/23177/ https://reviews.apache.org/r/23178/ https://reviews.apache.org/r/23179/ These are also available as a sequence of patches as the pull request https://github.com/apache/hbase/pull/1. Only the last patch hooks everything to the HBase build process (optionally) and hence I was thinking of squashing these separate patches into a single patch to be submitted for build. On Wed, Jul 9, 2014 at 11:32 AM, Nick Dimiduk ndimi...@gmail.com wrote: This ticket has only open subtasks, ie nothing in 'patch available'. I assume you mean HBASE-10168. We'll see about getting you some reviews, but you should also go about formatting the patch for buildbot. Also, since your 3 reviews are individually 100+k, you should consider breaking them into three separate tickets. my 2¢ -n On Mon, Jul 7, 2014 at 12:01 PM, Aditya adityakish...@gmail.com wrote: Sorry about that. Here is the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 On Mon, Jul 7, 2014 at 10:05 AM, Nick Dimiduk ndimi...@gmail.com wrote: Would you mind including the JIRA numbers along with the request? Thanks, Nick On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can
Re: DISCUSSION: 1.0.0
Looks like the regular patch command skips any binary included in patches. On Sat, Jul 19, 2014 at 1:37 AM, Aditya adityakish...@gmail.com wrote: Thanks for taking a look Ted! Looks like the second patch created with git diff excluded the Gradle wrapper JAR from the patch. I would generate a new one which includes this this jar. In the meantime, you should be able to use the first patch attached to the JIRA which is in git-am format and that would let you build. On Fri, Jul 18, 2014 at 3:40 PM, Ted Yu yuzhih...@gmail.com wrote: Nice work, Aditya. Looks like the hbase-native-client profile requires gradle ? [exec] Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain Will take a look at your patch. Cheers On Fri, Jul 18, 2014 at 3:08 PM, Aditya adityakish...@gmail.com wrote: As requested, I have attached a combined patch to the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 and submitted it to jenkins. Would be great if someone could take a look and provide feedback. Regards, aditya... On Wed, Jul 9, 2014 at 7:05 PM, Aditya adityakish...@gmail.com wrote: I was hoping to get some initial comments before attaching patches for the build boat. I have broken the entire code into 5 patch sets, layered in a sequnce, each focusing on a particular area (public headers/JNI implementation/Examples+unit test, etc) for the ease of review. https://reviews.apache.org/r/23175/ https://reviews.apache.org/r/23176/ https://reviews.apache.org/r/23177/ https://reviews.apache.org/r/23178/ https://reviews.apache.org/r/23179/ These are also available as a sequence of patches as the pull request https://github.com/apache/hbase/pull/1. Only the last patch hooks everything to the HBase build process (optionally) and hence I was thinking of squashing these separate patches into a single patch to be submitted for build. On Wed, Jul 9, 2014 at 11:32 AM, Nick Dimiduk ndimi...@gmail.com wrote: This ticket has only open subtasks, ie nothing in 'patch available'. I assume you mean HBASE-10168. We'll see about getting you some reviews, but you should also go about formatting the patch for buildbot. Also, since your 3 reviews are individually 100+k, you should consider breaking them into three separate tickets. my 2¢ -n On Mon, Jul 7, 2014 at 12:01 PM, Aditya adityakish...@gmail.com wrote: Sorry about that. Here is the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 On Mon, Jul 7, 2014 at 10:05 AM, Nick Dimiduk ndimi...@gmail.com wrote: Would you mind including the JIRA numbers along with the request? Thanks, Nick On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about
Re: DISCUSSION: 1.0.0
You may want to attach the wrapper jar to the JIRA directly. Cheers On Jul 19, 2014, at 1:52 AM, Aditya adityakish...@gmail.com wrote: Looks like the regular patch command skips any binary included in patches. On Sat, Jul 19, 2014 at 1:37 AM, Aditya adityakish...@gmail.com wrote: Thanks for taking a look Ted! Looks like the second patch created with git diff excluded the Gradle wrapper JAR from the patch. I would generate a new one which includes this this jar. In the meantime, you should be able to use the first patch attached to the JIRA which is in git-am format and that would let you build. On Fri, Jul 18, 2014 at 3:40 PM, Ted Yu yuzhih...@gmail.com wrote: Nice work, Aditya. Looks like the hbase-native-client profile requires gradle ? [exec] Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain Will take a look at your patch. Cheers On Fri, Jul 18, 2014 at 3:08 PM, Aditya adityakish...@gmail.com wrote: As requested, I have attached a combined patch to the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 and submitted it to jenkins. Would be great if someone could take a look and provide feedback. Regards, aditya... On Wed, Jul 9, 2014 at 7:05 PM, Aditya adityakish...@gmail.com wrote: I was hoping to get some initial comments before attaching patches for the build boat. I have broken the entire code into 5 patch sets, layered in a sequnce, each focusing on a particular area (public headers/JNI implementation/Examples+unit test, etc) for the ease of review. https://reviews.apache.org/r/23175/ https://reviews.apache.org/r/23176/ https://reviews.apache.org/r/23177/ https://reviews.apache.org/r/23178/ https://reviews.apache.org/r/23179/ These are also available as a sequence of patches as the pull request https://github.com/apache/hbase/pull/1. Only the last patch hooks everything to the HBase build process (optionally) and hence I was thinking of squashing these separate patches into a single patch to be submitted for build. On Wed, Jul 9, 2014 at 11:32 AM, Nick Dimiduk ndimi...@gmail.com wrote: This ticket has only open subtasks, ie nothing in 'patch available'. I assume you mean HBASE-10168. We'll see about getting you some reviews, but you should also go about formatting the patch for buildbot. Also, since your 3 reviews are individually 100+k, you should consider breaking them into three separate tickets. my 2¢ -n On Mon, Jul 7, 2014 at 12:01 PM, Aditya adityakish...@gmail.com wrote: Sorry about that. Here is the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 On Mon, Jul 7, 2014 at 10:05 AM, Nick Dimiduk ndimi...@gmail.com wrote: Would you mind including the JIRA numbers along with the request? Thanks, Nick On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909
Re: DISCUSSION: 1.0.0
As requested, I have attached a combined patch to the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 and submitted it to jenkins. Would be great if someone could take a look and provide feedback. Regards, aditya... On Wed, Jul 9, 2014 at 7:05 PM, Aditya adityakish...@gmail.com wrote: I was hoping to get some initial comments before attaching patches for the build boat. I have broken the entire code into 5 patch sets, layered in a sequnce, each focusing on a particular area (public headers/JNI implementation/Examples+unit test, etc) for the ease of review. https://reviews.apache.org/r/23175/ https://reviews.apache.org/r/23176/ https://reviews.apache.org/r/23177/ https://reviews.apache.org/r/23178/ https://reviews.apache.org/r/23179/ These are also available as a sequence of patches as the pull request https://github.com/apache/hbase/pull/1. Only the last patch hooks everything to the HBase build process (optionally) and hence I was thinking of squashing these separate patches into a single patch to be submitted for build. On Wed, Jul 9, 2014 at 11:32 AM, Nick Dimiduk ndimi...@gmail.com wrote: This ticket has only open subtasks, ie nothing in 'patch available'. I assume you mean HBASE-10168. We'll see about getting you some reviews, but you should also go about formatting the patch for buildbot. Also, since your 3 reviews are individually 100+k, you should consider breaking them into three separate tickets. my 2¢ -n On Mon, Jul 7, 2014 at 12:01 PM, Aditya adityakish...@gmail.com wrote: Sorry about that. Here is the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 On Mon, Jul 7, 2014 at 10:05 AM, Nick Dimiduk ndimi...@gmail.com wrote: Would you mind including the JIRA numbers along with the request? Thanks, Nick On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
This ticket has only open subtasks, ie nothing in 'patch available'. I assume you mean HBASE-10168. We'll see about getting you some reviews, but you should also go about formatting the patch for buildbot. Also, since your 3 reviews are individually 100+k, you should consider breaking them into three separate tickets. my 2¢ -n On Mon, Jul 7, 2014 at 12:01 PM, Aditya adityakish...@gmail.com wrote: Sorry about that. Here is the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 On Mon, Jul 7, 2014 at 10:05 AM, Nick Dimiduk ndimi...@gmail.com wrote: Would you mind including the JIRA numbers along with the request? Thanks, Nick On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
I was hoping to get some initial comments before attaching patches for the build boat. I have broken the entire code into 5 patch sets, layered in a sequnce, each focusing on a particular area (public headers/JNI implementation/Examples+unit test, etc) for the ease of review. https://reviews.apache.org/r/23175/ https://reviews.apache.org/r/23176/ https://reviews.apache.org/r/23177/ https://reviews.apache.org/r/23178/ https://reviews.apache.org/r/23179/ These are also available as a sequence of patches as the pull request https://github.com/apache/hbase/pull/1. Only the last patch hooks everything to the HBase build process (optionally) and hence I was thinking of squashing these separate patches into a single patch to be submitted for build. On Wed, Jul 9, 2014 at 11:32 AM, Nick Dimiduk ndimi...@gmail.com wrote: This ticket has only open subtasks, ie nothing in 'patch available'. I assume you mean HBASE-10168. We'll see about getting you some reviews, but you should also go about formatting the patch for buildbot. Also, since your 3 reviews are individually 100+k, you should consider breaking them into three separate tickets. my 2¢ -n On Mon, Jul 7, 2014 at 12:01 PM, Aditya adityakish...@gmail.com wrote: Sorry about that. Here is the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 On Mon, Jul 7, 2014 at 10:05 AM, Nick Dimiduk ndimi...@gmail.com wrote: Would you mind including the JIRA numbers along with the request? Thanks, Nick On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
Hi Aditya, I think it would be good to have a native client API in 1.0. I can look at the patches when back in the office later this week. On Jul 7, 2014, at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
Would you mind including the JIRA numbers along with the request? Thanks, Nick On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
The following is another candidate for 1.0.0 release HBASE-11447 Proposal for a generic transaction API for HBase Cheers On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
I thought the scope of HBASE-11447 is to review and refine proposed design, not to track the implementation of the feature? -Mikhail 2014-07-07 10:40 GMT-07:00 Andrew Purtell andrew.purt...@gmail.com: That is not a realistic proposal for 1.0 as far as I can see. There is only a tentative design spec on that issue. I assume we are sticking with Enis' earlier stated intent to get 1.0 out soon. On Jul 7, 2014, at 10:30 AM, Ted Yu yuzhih...@gmail.com wrote: The following is another candidate for 1.0.0 release HBASE-11447 Proposal for a generic transaction API for HBase Cheers On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
I think it is fair to say for both C API and transaction API that it should be ok to include them if they are ready before the time to cut the 1.0 RC. If not, I do not want to hold the release waiting for those. Enis On Mon, Jul 7, 2014 at 10:50 AM, Andrew Purtell andrew.purt...@gmail.com wrote: And by definition of that being a proposed design, there is no implementation to track (yet). There being no implementation, it's hard to see how it makes a 1.0 happening soon. Don't misunderstand me to be opposed to the idea or feature, far from it. I don't think Ted did anyone a service bringing it up in the context of 1.0. Already we are on some kind of detour from discussing things where there are patches available. On Jul 7, 2014, at 10:43 AM, Mikhail Antonov olorinb...@gmail.com wrote: I thought the scope of HBASE-11447 is to review and refine proposed design, not to track the implementation of the feature? -Mikhail 2014-07-07 10:40 GMT-07:00 Andrew Purtell andrew.purt...@gmail.com: That is not a realistic proposal for 1.0 as far as I can see. There is only a tentative design spec on that issue. I assume we are sticking with Enis' earlier stated intent to get 1.0 out soon. On Jul 7, 2014, at 10:30 AM, Ted Yu yuzhih...@gmail.com wrote: The following is another candidate for 1.0.0 release HBASE-11447 Proposal for a generic transaction API for HBase Cheers On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
And by definition of that being a proposed design, there is no implementation to track (yet). There being no implementation, it's hard to see how it makes a 1.0 happening soon. Don't misunderstand me to be opposed to the idea or feature, far from it. I don't think Ted did anyone a service bringing it up in the context of 1.0. Already we are on some kind of detour from discussing things where there are patches available. On Jul 7, 2014, at 10:43 AM, Mikhail Antonov olorinb...@gmail.com wrote: I thought the scope of HBASE-11447 is to review and refine proposed design, not to track the implementation of the feature? -Mikhail 2014-07-07 10:40 GMT-07:00 Andrew Purtell andrew.purt...@gmail.com: That is not a realistic proposal for 1.0 as far as I can see. There is only a tentative design spec on that issue. I assume we are sticking with Enis' earlier stated intent to get 1.0 out soon. On Jul 7, 2014, at 10:30 AM, Ted Yu yuzhih...@gmail.com wrote: The following is another candidate for 1.0.0 release HBASE-11447 Proposal for a generic transaction API for HBase Cheers On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
Sorry about that. Here is the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015 On Mon, Jul 7, 2014 at 10:05 AM, Nick Dimiduk ndimi...@gmail.com wrote: Would you mind including the JIRA numbers along with the request? Thanks, Nick On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
We're on the same page, Enis. Cheers On Mon, Jul 7, 2014 at 10:56 AM, Enis Söztutar e...@apache.org wrote: I think it is fair to say for both C API and transaction API that it should be ok to include them if they are ready before the time to cut the 1.0 RC. If not, I do not want to hold the release waiting for those. Enis On Mon, Jul 7, 2014 at 10:50 AM, Andrew Purtell andrew.purt...@gmail.com wrote: And by definition of that being a proposed design, there is no implementation to track (yet). There being no implementation, it's hard to see how it makes a 1.0 happening soon. Don't misunderstand me to be opposed to the idea or feature, far from it. I don't think Ted did anyone a service bringing it up in the context of 1.0. Already we are on some kind of detour from discussing things where there are patches available. On Jul 7, 2014, at 10:43 AM, Mikhail Antonov olorinb...@gmail.com wrote: I thought the scope of HBASE-11447 is to review and refine proposed design, not to track the implementation of the feature? -Mikhail 2014-07-07 10:40 GMT-07:00 Andrew Purtell andrew.purt...@gmail.com: That is not a realistic proposal for 1.0 as far as I can see. There is only a tentative design spec on that issue. I assume we are sticking with Enis' earlier stated intent to get 1.0 out soon. On Jul 7, 2014, at 10:30 AM, Ted Yu yuzhih...@gmail.com wrote: The following is another candidate for 1.0.0 release HBASE-11447 Proposal for a generic transaction API for HBase Cheers On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
I agree. These are fine things we could get in 1.1 or 1.2 release if they don't break compat and otherwise a 2.0 release timeframe. Jon. On Mon, Jul 7, 2014 at 10:56 AM, Enis Söztutar e...@apache.org wrote: I think it is fair to say for both C API and transaction API that it should be ok to include them if they are ready before the time to cut the 1.0 RC. If not, I do not want to hold the release waiting for those. Enis On Mon, Jul 7, 2014 at 10:50 AM, Andrew Purtell andrew.purt...@gmail.com wrote: And by definition of that being a proposed design, there is no implementation to track (yet). There being no implementation, it's hard to see how it makes a 1.0 happening soon. Don't misunderstand me to be opposed to the idea or feature, far from it. I don't think Ted did anyone a service bringing it up in the context of 1.0. Already we are on some kind of detour from discussing things where there are patches available. On Jul 7, 2014, at 10:43 AM, Mikhail Antonov olorinb...@gmail.com wrote: I thought the scope of HBASE-11447 is to review and refine proposed design, not to track the implementation of the feature? -Mikhail 2014-07-07 10:40 GMT-07:00 Andrew Purtell andrew.purt...@gmail.com: That is not a realistic proposal for 1.0 as far as I can see. There is only a tentative design spec on that issue. I assume we are sticking with Enis' earlier stated intent to get 1.0 out soon. On Jul 7, 2014, at 10:30 AM, Ted Yu yuzhih...@gmail.com wrote: The following is another candidate for 1.0.0 release HBASE-11447 Proposal for a generic transaction API for HBase Cheers On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote: Do we want to have the C APIs part of 1.0.0 release. I had posted few patches and a set of review request sometime last week. On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote: On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov -- Thanks, Michael Antonov -- // Jonathan Hsieh (shay) // HBase Tech Lead, Software Engineer, Cloudera // j...@cloudera.com // @jmhsieh
Re: DISCUSSION: 1.0.0
On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com wrote: Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? Sounds good. HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Not sure whether we can make it fully backwards compatible with 1.0 clients. I guess we will see when the patches are done. Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. Thoughts? Thanks, Mikhail 2014-06-02 13:00 GMT-07:00 Jimmy Xiang jxi...@cloudera.com: For ZK-less assignment, I prefer it to go in before 1.0. It's backward compatible and rolling-upgradable. I am doing more testing now. Thanks, Jimmy On Mon, Jun 2, 2014 at 12:54 PM, Mikhail Antonov olorinb...@gmail.com wrote: On ZK abstraction... What date tentatively we're talking as a release date for 1.0.0? I think Enis is right that technically as long as we're just changing private interfaces, we can do part in one release and the rest in subsequent one, yet also I agree with Andrew that having ZK partially-separated is a bit odd. On other consideration is that we can't do anything non-rolling-upgradable before 1.0 is out, IIUC. I would say - let's aim to have not-too-intrusive refactoring work as part of 1.0 (non-intrusive means - not changing control flow through ZK or data stored in ZooKeeper). If we end up in in situation when ZK abstraction is on the critical part for release - let's reduce the scope and leave certain areas (like replication queues) for the subsequent release. And from 1.0 onwards - we can start redesigning the way we store shared state and tackle multiple active master/region replicas. Thoughts? Jimmy - what about ZK-less assignment, will it go in before 1.0? -Mikhail 2014-06-02 11:49 GMT-07:00 Andrew Purtell apurt...@apache.org: On Mon, Jun 2, 2014 at 11:43 AM, Enis Söztutar enis@gmail.com wrote: On Mon, Jun 2, 2014 at 10:25 AM, Andrew Purtell apurt...@apache.org wrote: An email from JIRA reminds me that we should also have the ZooKeeper related refactoring complete in 1.0 before releasing it. That work is pretty far along and needs all bits in place to be useful. Agreed that it will be good to get this completed. However, they are mostly internal interfaces and I am not sure whether all the changes required will be done in time. We can continue on this even after 1.0, no? I think it would be weird to have a release where we are partially on the way to plugging ZooKeeper in and out but not all the way, so that's not actually possible. -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- Thanks, Michael Antonov -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack
Re: DISCUSSION: 1.0.0
Moved ZK watcher listener subtask out of scope HBASE-10909. Enis - with that, I guess HBASE-10909 can be marked in branch-1? HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase client from ZK (mostly it would be post-1.0 work). Thanks, Mikhail 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net: On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com wrote: Guys, getting back to ZK abstraction work w.r.t. release 1.0 and thereafter, some status update. So as we're getting closer to complete HBASE-10909, it looks like the steps may be like this: - there are 2 subtasks out there not closed yet, one of which is about log splitting (and Sergey S has submitted a patch for review), another is abstraction of ZK watcher (this is what I've been working on in the background; but after sketching the patch it seems like without being able to modify the control flows and some changes in the module structure, it'd be a lot of scaffolding code not really simplifying current code). So I'd propose to descope abstraction of ZK watcher jira (HBASE-11073), namely: convert it to top-level JIRA and continue to work on it separately; rename HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as soon as log splitting jira is completed. This way HBASE-10909 fits to branch-1. Sounds good to me. - secondly, in the discussion to the CatalogTracker patch, we started talking about modifying client to not know about ZK, but rather keep the location of current masters and talk to them using RPC calls. This work can not go into branch-1, as it involves invasive changes in client including new RPC. As I understand the branching schema now, those changes can go to master branch, we just don't merge them to branch-1, and depending on their completeness we can pull them to 1.1 release or so. You have it right Mikhail. St.Ack -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
1.0.0 has been going on for a while now. Master has a bunch of good stuff in it . What are we thinking of as a release date for the first 0.99.0 and for 1.0.0 itself? Thanks, St.Ack On Thu, Jan 2, 2014 at 11:26 AM, Stack st...@duboce.net wrote: Andrew is talking of the first 0.98RC being imminent. Time to start in on the release that will follow 0.98.x. We seem to all be good with calling it 1.0.0. Speak up if you think different. (I just added a 1.0.0 version to JIRA). + What should 1.0.0 have in it beyond what is in 0.98. + Why can't 1.0.0 just be 0.98.0, or 0.98.1 altogether? + When should it come out? I'm thinking soon after 0.98. Feb/March? (Presuming 0.98 ships in Jan). + Who should RM it? (I could but perhaps others are interested). What else should we consider achieving the state of 1.0.0ness? Happy New Year all, St.Ack
Re: DISCUSSION: 1.0.0
HBASE-10856 has 8 open subtasks, 6 of which are not assigned. Four other JIRAs (HBASE-9864, HBASE-11122, HBASE-11124, and HBASE-11225) are incorporated by reference and are open. Those could be dropped. All but HBASE-11122 represent significant work. My guess based on the lack of activity on the 1.0 JIRA is it will be open for a long time without much attention. Perhaps we can instead move much to a new JIRA serving as an umbrella for 1.1 and call 1.0 as imminent. Merge HBASE-10070 into trunk - if the vote passes - and then only keep the issues for updating documentation and testing rolling restart / compat with 0.98? On Mon, Jun 2, 2014 at 10:11 AM, Stack st...@duboce.net wrote: 1.0.0 has been going on for a while now. Master has a bunch of good stuff in it . What are we thinking of as a release date for the first 0.99.0 and for 1.0.0 itself? Thanks, St.Ack On Thu, Jan 2, 2014 at 11:26 AM, Stack st...@duboce.net wrote: Andrew is talking of the first 0.98RC being imminent. Time to start in on the release that will follow 0.98.x. We seem to all be good with calling it 1.0.0. Speak up if you think different. (I just added a 1.0.0 version to JIRA). + What should 1.0.0 have in it beyond what is in 0.98. + Why can't 1.0.0 just be 0.98.0, or 0.98.1 altogether? + When should it come out? I'm thinking soon after 0.98. Feb/March? (Presuming 0.98 ships in Jan). + Who should RM it? (I could but perhaps others are interested). What else should we consider achieving the state of 1.0.0ness? Happy New Year all, St.Ack -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
Re: DISCUSSION: 1.0.0
An email from JIRA reminds me that we should also have the ZooKeeper related refactoring complete in 1.0 before releasing it. That work is pretty far along and needs all bits in place to be useful. On Mon, Jun 2, 2014 at 10:23 AM, Andrew Purtell apurt...@apache.org wrote: HBASE-10856 has 8 open subtasks, 6 of which are not assigned. Four other JIRAs (HBASE-9864, HBASE-11122, HBASE-11124, and HBASE-11225) are incorporated by reference and are open. Those could be dropped. All but HBASE-11122 represent significant work. My guess based on the lack of activity on the 1.0 JIRA is it will be open for a long time without much attention. Perhaps we can instead move much to a new JIRA serving as an umbrella for 1.1 and call 1.0 as imminent. Merge HBASE-10070 into trunk - if the vote passes - and then only keep the issues for updating documentation and testing rolling restart / compat with 0.98? On Mon, Jun 2, 2014 at 10:11 AM, Stack st...@duboce.net wrote: 1.0.0 has been going on for a while now. Master has a bunch of good stuff in it . What are we thinking of as a release date for the first 0.99.0 and for 1.0.0 itself? Thanks, St.Ack On Thu, Jan 2, 2014 at 11:26 AM, Stack st...@duboce.net wrote: Andrew is talking of the first 0.98RC being imminent. Time to start in on the release that will follow 0.98.x. We seem to all be good with calling it 1.0.0. Speak up if you think different. (I just added a 1.0.0 version to JIRA). + What should 1.0.0 have in it beyond what is in 0.98. + Why can't 1.0.0 just be 0.98.0, or 0.98.1 altogether? + When should it come out? I'm thinking soon after 0.98. Feb/March? (Presuming 0.98 ships in Jan). + Who should RM it? (I could but perhaps others are interested). What else should we consider achieving the state of 1.0.0ness? Happy New Year all, St.Ack -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
Re: DISCUSSION: 1.0.0
On Mon, Jun 2, 2014 at 10:25 AM, Andrew Purtell apurt...@apache.org wrote: An email from JIRA reminds me that we should also have the ZooKeeper related refactoring complete in 1.0 before releasing it. That work is pretty far along and needs all bits in place to be useful. Agreed that it will be good to get this completed. However, they are mostly internal interfaces and I am not sure whether all the changes required will be done in time. We can continue on this even after 1.0, no? On Mon, Jun 2, 2014 at 10:23 AM, Andrew Purtell apurt...@apache.org wrote: HBASE-10856 has 8 open subtasks, 6 of which are not assigned. Four other JIRAs (HBASE-9864, HBASE-11122, HBASE-11124, and HBASE-11225) are incorporated by reference and are open. Those could be dropped. All but HBASE-11122 represent significant work. My guess based on the lack of activity on the 1.0 JIRA is it will be open for a long time without much attention. Perhaps we can instead move much to a new JIRA serving as an umbrella for 1.1 and call 1.0 as imminent. Merge HBASE-10070 into trunk - if the vote passes - and then only keep the issues for updating documentation and testing rolling restart / compat with 0.98? I would like to get HBASE-10070 merged. Let me start the VOTE, now that the DISCUSSION thread died down. On Mon, Jun 2, 2014 at 10:11 AM, Stack st...@duboce.net wrote: 1.0.0 has been going on for a while now. Master has a bunch of good stuff in it . What are we thinking of as a release date for the first 0.99.0 and for 1.0.0 itself? I think we can cut 0.99 in a couple of weeks. I was aiming Aug timeframe for an eventual 1.0 release. After that we will have a branch that we can selectively include features only needed for the release. Thanks, St.Ack On Thu, Jan 2, 2014 at 11:26 AM, Stack st...@duboce.net wrote: Andrew is talking of the first 0.98RC being imminent. Time to start in on the release that will follow 0.98.x. We seem to all be good with calling it 1.0.0. Speak up if you think different. (I just added a 1.0.0 version to JIRA). + What should 1.0.0 have in it beyond what is in 0.98. + Why can't 1.0.0 just be 0.98.0, or 0.98.1 altogether? + When should it come out? I'm thinking soon after 0.98. Feb/March? (Presuming 0.98 ships in Jan). + Who should RM it? (I could but perhaps others are interested). What else should we consider achieving the state of 1.0.0ness? Happy New Year all, St.Ack -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
Re: DISCUSSION: 1.0.0
On Mon, Jun 2, 2014 at 11:43 AM, Enis Söztutar enis@gmail.com wrote: On Mon, Jun 2, 2014 at 10:25 AM, Andrew Purtell apurt...@apache.org wrote: An email from JIRA reminds me that we should also have the ZooKeeper related refactoring complete in 1.0 before releasing it. That work is pretty far along and needs all bits in place to be useful. Agreed that it will be good to get this completed. However, they are mostly internal interfaces and I am not sure whether all the changes required will be done in time. We can continue on this even after 1.0, no? I think it would be weird to have a release where we are partially on the way to plugging ZooKeeper in and out but not all the way, so that's not actually possible. -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
Re: DISCUSSION: 1.0.0
On ZK abstraction... What date tentatively we're talking as a release date for 1.0.0? I think Enis is right that technically as long as we're just changing private interfaces, we can do part in one release and the rest in subsequent one, yet also I agree with Andrew that having ZK partially-separated is a bit odd. On other consideration is that we can't do anything non-rolling-upgradable before 1.0 is out, IIUC. I would say - let's aim to have not-too-intrusive refactoring work as part of 1.0 (non-intrusive means - not changing control flow through ZK or data stored in ZooKeeper). If we end up in in situation when ZK abstraction is on the critical part for release - let's reduce the scope and leave certain areas (like replication queues) for the subsequent release. And from 1.0 onwards - we can start redesigning the way we store shared state and tackle multiple active master/region replicas. Thoughts? Jimmy - what about ZK-less assignment, will it go in before 1.0? -Mikhail 2014-06-02 11:49 GMT-07:00 Andrew Purtell apurt...@apache.org: On Mon, Jun 2, 2014 at 11:43 AM, Enis Söztutar enis@gmail.com wrote: On Mon, Jun 2, 2014 at 10:25 AM, Andrew Purtell apurt...@apache.org wrote: An email from JIRA reminds me that we should also have the ZooKeeper related refactoring complete in 1.0 before releasing it. That work is pretty far along and needs all bits in place to be useful. Agreed that it will be good to get this completed. However, they are mostly internal interfaces and I am not sure whether all the changes required will be done in time. We can continue on this even after 1.0, no? I think it would be weird to have a release where we are partially on the way to plugging ZooKeeper in and out but not all the way, so that's not actually possible. -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
For ZK-less assignment, I prefer it to go in before 1.0. It's backward compatible and rolling-upgradable. I am doing more testing now. Thanks, Jimmy On Mon, Jun 2, 2014 at 12:54 PM, Mikhail Antonov olorinb...@gmail.com wrote: On ZK abstraction... What date tentatively we're talking as a release date for 1.0.0? I think Enis is right that technically as long as we're just changing private interfaces, we can do part in one release and the rest in subsequent one, yet also I agree with Andrew that having ZK partially-separated is a bit odd. On other consideration is that we can't do anything non-rolling-upgradable before 1.0 is out, IIUC. I would say - let's aim to have not-too-intrusive refactoring work as part of 1.0 (non-intrusive means - not changing control flow through ZK or data stored in ZooKeeper). If we end up in in situation when ZK abstraction is on the critical part for release - let's reduce the scope and leave certain areas (like replication queues) for the subsequent release. And from 1.0 onwards - we can start redesigning the way we store shared state and tackle multiple active master/region replicas. Thoughts? Jimmy - what about ZK-less assignment, will it go in before 1.0? -Mikhail 2014-06-02 11:49 GMT-07:00 Andrew Purtell apurt...@apache.org: On Mon, Jun 2, 2014 at 11:43 AM, Enis Söztutar enis@gmail.com wrote: On Mon, Jun 2, 2014 at 10:25 AM, Andrew Purtell apurt...@apache.org wrote: An email from JIRA reminds me that we should also have the ZooKeeper related refactoring complete in 1.0 before releasing it. That work is pretty far along and needs all bits in place to be useful. Agreed that it will be good to get this completed. However, they are mostly internal interfaces and I am not sure whether all the changes required will be done in time. We can continue on this even after 1.0, no? I think it would be weird to have a release where we are partially on the way to plugging ZooKeeper in and out but not all the way, so that's not actually possible. -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- Thanks, Michael Antonov
Re: DISCUSSION: 1.0.0
The following thread brought attention to HBASE-9206 'namespace permissions' : http://search-hadoop.com/m/DHED49RMDk/enable%252Fdisable+table+permissionsubj=enable+disable+table+permission This would help user implement multi-tenancy in a shared cluster. Cheers On Tue, Feb 11, 2014 at 12:43 PM, Jonathan Hsieh j...@cloudera.com wrote: Lily's indexer uses hbase private apis in the replication portion of code. we may want to expose and make promises on the replication api/wire to make their lives easier and to enable cross version replication for future hbases. Jon. On Tue, Feb 11, 2014 at 11:21 AM, Nick Dimiduk ndimi...@gmail.com wrote: FYI, Lily's Indexer [0] also makes use of this level of integration into HBase. Others? [0]: https://github.com/NGDATA/hbase-indexer On Tue, Feb 11, 2014 at 12:09 PM, Enis Söztutar enis@gmail.com wrote: This is good discussion, I mentioned this in phoenix-dev that we need to repurpose coprocessors, and just give them put / get kind of injection points, but not allow co-processors to tap into log / compaction, etc. Those will be better served by explicitly defined plugins (like the recent StorageEngine, HLog, etc) that we explicitly allow injecting code with some API stability (though probably still evolving at a fast pace). It would be good to have an idea of what classes / methods, phoenix and similar depend on HBase right now. Enis On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong jzh...@hortonworks.com wrote: We need to revisit which interfaces should be marked as real private because coprocessors allow custom applications to accesses internal states. Current private interface such as KeyValue, WALEdit, Hregion, HLogKey, Store etc is may used by custom coprocessor applications so any public method signature change in those private interfaces possibliy breaks custom coprocessor implementations. -Jeffrey On 2/10/14 4:59 PM, lars hofhansl la...@apache.org wrote: True. But we can clearly annotate what is public and what is not. In HRegion we'd have to do it per method. It's probably easier to extract the public interface out into a Interface. (similar to Store and HStore) HBASE-4207 solves a slightly different problem. Here I want to have access to some HBase internals for performance, but I want to know what is public and stable and what I should not touch (thinking about things like Phoenix). -- Lars From: Andrew Purtell apurt...@apache.org To: dev@hbase.apache.org dev@hbase.apache.org Sent: Monday, February 10, 2014 4:46 PM Subject: Re: DISCUSSION: 1.0.0 On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl la...@apache.org wrote: + known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. We started this with Environment and the HTable wrapper as an example of how to wrap an interface for CPs. At the end of the day we can't stop a coprocessor from accessing internal objects and calling their methods directly until HBASE-4047. (Maybe that makes the 1.0 list?) Related, paring down the coprocessor interfaces to only intercept RPC based actions and move everything else to plugins. -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- // Jonathan Hsieh (shay) // HBase Tech Lead, Software Engineer, Cloudera // j...@cloudera.com // @jmhsieh
Re: DISCUSSION: 1.0.0
Thanks Ted. It makes sense to fix it before 1.0 if we can. Enis On Sun, Mar 2, 2014 at 8:54 PM, Ted Yu yuzhih...@gmail.com wrote: The following thread brought attention to HBASE-9206 'namespace permissions' : http://search-hadoop.com/m/DHED49RMDk/enable%252Fdisable+table+permissionsubj=enable+disable+table+permission This would help user implement multi-tenancy in a shared cluster. Cheers On Tue, Feb 11, 2014 at 12:43 PM, Jonathan Hsieh j...@cloudera.com wrote: Lily's indexer uses hbase private apis in the replication portion of code. we may want to expose and make promises on the replication api/wire to make their lives easier and to enable cross version replication for future hbases. Jon. On Tue, Feb 11, 2014 at 11:21 AM, Nick Dimiduk ndimi...@gmail.com wrote: FYI, Lily's Indexer [0] also makes use of this level of integration into HBase. Others? [0]: https://github.com/NGDATA/hbase-indexer On Tue, Feb 11, 2014 at 12:09 PM, Enis Söztutar enis@gmail.com wrote: This is good discussion, I mentioned this in phoenix-dev that we need to repurpose coprocessors, and just give them put / get kind of injection points, but not allow co-processors to tap into log / compaction, etc. Those will be better served by explicitly defined plugins (like the recent StorageEngine, HLog, etc) that we explicitly allow injecting code with some API stability (though probably still evolving at a fast pace). It would be good to have an idea of what classes / methods, phoenix and similar depend on HBase right now. Enis On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong jzh...@hortonworks.com wrote: We need to revisit which interfaces should be marked as real private because coprocessors allow custom applications to accesses internal states. Current private interface such as KeyValue, WALEdit, Hregion, HLogKey, Store etc is may used by custom coprocessor applications so any public method signature change in those private interfaces possibliy breaks custom coprocessor implementations. -Jeffrey On 2/10/14 4:59 PM, lars hofhansl la...@apache.org wrote: True. But we can clearly annotate what is public and what is not. In HRegion we'd have to do it per method. It's probably easier to extract the public interface out into a Interface. (similar to Store and HStore) HBASE-4207 solves a slightly different problem. Here I want to have access to some HBase internals for performance, but I want to know what is public and stable and what I should not touch (thinking about things like Phoenix). -- Lars From: Andrew Purtell apurt...@apache.org To: dev@hbase.apache.org dev@hbase.apache.org Sent: Monday, February 10, 2014 4:46 PM Subject: Re: DISCUSSION: 1.0.0 On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl la...@apache.org wrote: + known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. We started this with Environment and the HTable wrapper as an example of how to wrap an interface for CPs. At the end of the day we can't stop a coprocessor from accessing internal objects and calling their methods directly until HBASE-4047. (Maybe that makes the 1.0 list?) Related, paring down the coprocessor interfaces to only intercept RPC based actions and move everything else to plugins. -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- // Jonathan Hsieh (shay) // HBase Tech Lead, Software Engineer, Cloudera // j...@cloudera.com // @jmhsieh
Re: DISCUSSION: 1.0.0
'Repurpose' might not be the way I would put it. Coprocessors were and are a means for internal server extension by mixin. The original problem we solved was needing to subclass HRegionServer and other classes to extend core HBase functions, but having more than one otherwise orthogonal extension that users want to use. Now we can mix in multiple extensions with a framework that has some simple rules for cooperation between the extensions. We return to the earlier state of affairs with modules. Sure, we can plug in an alternate behavior with a module that subclasses and extends the default, say flush strategy, but we can't then instantiate multiple modules into the same slot, both subclassing the same base but doing different things. On Feb 11, 2014, at 11:09 AM, Enis Söztutar enis@gmail.com wrote: This is good discussion, I mentioned this in phoenix-dev that we need to repurpose coprocessors, and just give them put / get kind of injection points, but not allow co-processors to tap into log / compaction, etc. Those will be better served by explicitly defined plugins (like the recent StorageEngine, HLog, etc) that we explicitly allow injecting code with some API stability (though probably still evolving at a fast pace). It would be good to have an idea of what classes / methods, phoenix and similar depend on HBase right now. Enis On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong jzh...@hortonworks.comwrote: We need to revisit which interfaces should be marked as real private because coprocessors allow custom applications to accesses internal states. Current private interface such as KeyValue, WALEdit, Hregion, HLogKey, Store etc is may used by custom coprocessor applications so any public method signature change in those private interfaces possibliy breaks custom coprocessor implementations. -Jeffrey On 2/10/14 4:59 PM, lars hofhansl la...@apache.org wrote: True. But we can clearly annotate what is public and what is not. In HRegion we'd have to do it per method. It's probably easier to extract the public interface out into a Interface. (similar to Store and HStore) HBASE-4207 solves a slightly different problem. Here I want to have access to some HBase internals for performance, but I want to know what is public and stable and what I should not touch (thinking about things like Phoenix). -- Lars From: Andrew Purtell apurt...@apache.org To: dev@hbase.apache.org dev@hbase.apache.org Sent: Monday, February 10, 2014 4:46 PM Subject: Re: DISCUSSION: 1.0.0 On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl la...@apache.org wrote: + known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. We started this with Environment and the HTable wrapper as an example of how to wrap an interface for CPs. At the end of the day we can't stop a coprocessor from accessing internal objects and calling their methods directly until HBASE-4047. (Maybe that makes the 1.0 list?) Related, paring down the coprocessor interfaces to only intercept RPC based actions and move everything else to plugins. -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: DISCUSSION: 1.0.0
'Repurpose' might not be the way I would put it. Coprocessors were and are a means for internal server extension by mixin. The original problem we solved was needing to subclass HRegionServer and other classes to extend core HBase functions, but having more than one otherwise orthogonal extension that users want to use. Now we can mix in multiple extensions with a framework that has some simple rules for cooperation between the extensions. We return to the earlier state of affairs with modules. Sure, we can plug in an alternate behavior with a module that subclasses and extends the default, say flush strategy, but we can't then instantiate multiple modules into the same slot, both subclassing the same base but doing different things. I agree the ability to compose coprocessors in order to extend behavior is a key capability that we should not throw out. I think the current Observer APIs could probably do with a bit of reorganization to make them a little more accessible and comprehensible. I think there is also an emerging need to see if we can define some subset of these APIs that we can stabilize for easier public consumption, while keeping the rest of the APIs free to evolve as needed as HBase internals change (since these are an extension mechanism for internal behaviors). I'm not sure we've really seen enough commonality emerge yet to say what those APIs are though. We could try to define the public subset as those involved in client requests, but flush and compaction, for example, can also be triggered by client requests. And my own use of coprocessor APIs lately has been focused on overriding the flush and compaction behaviors, not on client requests. I think the best place to start is by breaking up some of the current APIs, grouping them around behaviors or areas of functionality. Whether we call some of these coprocessors and others plugins is a question of branding. I do think it's important to figure out which we can stabilize and offer longer term contracts for. But whatever we call them, I strongly agree that we should maintain the mixin / composition approach and not return to a simple fixed inheritance scheme.
Re: DISCUSSION: 1.0.0
On Wed, Feb 12, 2014 at 12:46 PM, Gary Helmling ghelml...@gmail.com wrote: 'Repurpose' might not be the way I would put it. Coprocessors were and are a means for internal server extension by mixin. The original problem we solved was needing to subclass HRegionServer and other classes to extend core HBase functions, but having more than one otherwise orthogonal extension that users want to use. Now we can mix in multiple extensions with a framework that has some simple rules for cooperation between the extensions. We return to the earlier state of affairs with modules. Sure, we can plug in an alternate behavior with a module that subclasses and extends the default, say flush strategy, but we can't then instantiate multiple modules into the same slot, both subclassing the same base but doing different things. I agree the ability to compose coprocessors in order to extend behavior is a key capability that we should not throw out. I think the current Observer APIs could probably do with a bit of reorganization to make them a little more accessible and comprehensible. I think there is also an emerging need to see if we can define some subset of these APIs that we can stabilize for easier public consumption, while keeping the rest of the APIs free to evolve as needed as HBase internals change (since these are an extension mechanism for internal behaviors). I'm not sure we've really seen enough commonality emerge yet to say what those APIs are though. We could try to define the public subset as those involved in client requests, but flush and compaction, for example, can also be triggered by client requests. And my own use of coprocessor APIs lately has been focused on overriding the flush and compaction behaviors, not on client requests. I think the best place to start is by breaking up some of the current APIs, grouping them around behaviors or areas of functionality. Whether we call some of these coprocessors and others plugins is a question of branding. I do think it's important to figure out which we can stabilize and offer longer term contracts for. But whatever we call them, I strongly agree that we should maintain the mixin / composition approach and not return to a simple fixed inheritance scheme. I've always considered coprocessors to be the kernel modules of the HBase world. They give you way more power than user-space programming, but come with the cautions that if you make a mistake, you'll crash your whole system or trigger unexpected behavior. Given that, I don't think we should really be spending too much effort on coprocessor API stability. If we make this a requirement, it can hamper the ability of the HBase core developers to make good changes which really improve the system. I don't think we're at the level of maturity as a project where this is the right tradeoff, as of yet. For what it's worth, the Linux kernel module API is also not stable/compatible between versions. This document is a good read: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/stable_api_nonsense.txt I do think we should seek to keep the interfaces stable through *patch* level releases -- a bug fix shouldn't break a coprocessor API. But between minor releases that add new features, it seems like an unnecessary restriction. -Todd -- Todd Lipcon Software Engineer, Cloudera
Re: DISCUSSION: 1.0.0
This is no coincidence, Linux kernel modules (specifically the security ones, and what enables them) were some direct inspiration. :-) I may have said this before but I think stable interfaces for coprocessors comes into play only once we are doing HBASE-4047, and even then I would expect an eventing model where the vocabulary evolves from release to release. On Wed, Feb 12, 2014 at 1:22 PM, Todd Lipcon t...@cloudera.com wrote: On Wed, Feb 12, 2014 at 12:46 PM, Gary Helmling ghelml...@gmail.com wrote: 'Repurpose' might not be the way I would put it. Coprocessors were and are a means for internal server extension by mixin. The original problem we solved was needing to subclass HRegionServer and other classes to extend core HBase functions, but having more than one otherwise orthogonal extension that users want to use. Now we can mix in multiple extensions with a framework that has some simple rules for cooperation between the extensions. We return to the earlier state of affairs with modules. Sure, we can plug in an alternate behavior with a module that subclasses and extends the default, say flush strategy, but we can't then instantiate multiple modules into the same slot, both subclassing the same base but doing different things. I agree the ability to compose coprocessors in order to extend behavior is a key capability that we should not throw out. I think the current Observer APIs could probably do with a bit of reorganization to make them a little more accessible and comprehensible. I think there is also an emerging need to see if we can define some subset of these APIs that we can stabilize for easier public consumption, while keeping the rest of the APIs free to evolve as needed as HBase internals change (since these are an extension mechanism for internal behaviors). I'm not sure we've really seen enough commonality emerge yet to say what those APIs are though. We could try to define the public subset as those involved in client requests, but flush and compaction, for example, can also be triggered by client requests. And my own use of coprocessor APIs lately has been focused on overriding the flush and compaction behaviors, not on client requests. I think the best place to start is by breaking up some of the current APIs, grouping them around behaviors or areas of functionality. Whether we call some of these coprocessors and others plugins is a question of branding. I do think it's important to figure out which we can stabilize and offer longer term contracts for. But whatever we call them, I strongly agree that we should maintain the mixin / composition approach and not return to a simple fixed inheritance scheme. I've always considered coprocessors to be the kernel modules of the HBase world. They give you way more power than user-space programming, but come with the cautions that if you make a mistake, you'll crash your whole system or trigger unexpected behavior. Given that, I don't think we should really be spending too much effort on coprocessor API stability. If we make this a requirement, it can hamper the ability of the HBase core developers to make good changes which really improve the system. I don't think we're at the level of maturity as a project where this is the right tradeoff, as of yet. For what it's worth, the Linux kernel module API is also not stable/compatible between versions. This document is a good read: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/stable_api_nonsense.txt I do think we should seek to keep the interfaces stable through *patch* level releases -- a bug fix shouldn't break a coprocessor API. But between minor releases that add new features, it seems like an unnecessary restriction. -Todd -- Todd Lipcon Software Engineer, Cloudera -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
Re: DISCUSSION: 1.0.0
This is good discussion, I mentioned this in phoenix-dev that we need to repurpose coprocessors, and just give them put / get kind of injection points, but not allow co-processors to tap into log / compaction, etc. Those will be better served by explicitly defined plugins (like the recent StorageEngine, HLog, etc) that we explicitly allow injecting code with some API stability (though probably still evolving at a fast pace). It would be good to have an idea of what classes / methods, phoenix and similar depend on HBase right now. Enis On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong jzh...@hortonworks.comwrote: We need to revisit which interfaces should be marked as real private because coprocessors allow custom applications to accesses internal states. Current private interface such as KeyValue, WALEdit, Hregion, HLogKey, Store etc is may used by custom coprocessor applications so any public method signature change in those private interfaces possibliy breaks custom coprocessor implementations. -Jeffrey On 2/10/14 4:59 PM, lars hofhansl la...@apache.org wrote: True. But we can clearly annotate what is public and what is not. In HRegion we'd have to do it per method. It's probably easier to extract the public interface out into a Interface. (similar to Store and HStore) HBASE-4207 solves a slightly different problem. Here I want to have access to some HBase internals for performance, but I want to know what is public and stable and what I should not touch (thinking about things like Phoenix). -- Lars From: Andrew Purtell apurt...@apache.org To: dev@hbase.apache.org dev@hbase.apache.org Sent: Monday, February 10, 2014 4:46 PM Subject: Re: DISCUSSION: 1.0.0 On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl la...@apache.org wrote: + known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. We started this with Environment and the HTable wrapper as an example of how to wrap an interface for CPs. At the end of the day we can't stop a coprocessor from accessing internal objects and calling their methods directly until HBASE-4047. (Maybe that makes the 1.0 list?) Related, paring down the coprocessor interfaces to only intercept RPC based actions and move everything else to plugins. -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: DISCUSSION: 1.0.0
FYI, Lily's Indexer [0] also makes use of this level of integration into HBase. Others? [0]: https://github.com/NGDATA/hbase-indexer On Tue, Feb 11, 2014 at 12:09 PM, Enis Söztutar enis@gmail.com wrote: This is good discussion, I mentioned this in phoenix-dev that we need to repurpose coprocessors, and just give them put / get kind of injection points, but not allow co-processors to tap into log / compaction, etc. Those will be better served by explicitly defined plugins (like the recent StorageEngine, HLog, etc) that we explicitly allow injecting code with some API stability (though probably still evolving at a fast pace). It would be good to have an idea of what classes / methods, phoenix and similar depend on HBase right now. Enis On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong jzh...@hortonworks.com wrote: We need to revisit which interfaces should be marked as real private because coprocessors allow custom applications to accesses internal states. Current private interface such as KeyValue, WALEdit, Hregion, HLogKey, Store etc is may used by custom coprocessor applications so any public method signature change in those private interfaces possibliy breaks custom coprocessor implementations. -Jeffrey On 2/10/14 4:59 PM, lars hofhansl la...@apache.org wrote: True. But we can clearly annotate what is public and what is not. In HRegion we'd have to do it per method. It's probably easier to extract the public interface out into a Interface. (similar to Store and HStore) HBASE-4207 solves a slightly different problem. Here I want to have access to some HBase internals for performance, but I want to know what is public and stable and what I should not touch (thinking about things like Phoenix). -- Lars From: Andrew Purtell apurt...@apache.org To: dev@hbase.apache.org dev@hbase.apache.org Sent: Monday, February 10, 2014 4:46 PM Subject: Re: DISCUSSION: 1.0.0 On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl la...@apache.org wrote: + known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. We started this with Environment and the HTable wrapper as an example of how to wrap an interface for CPs. At the end of the day we can't stop a coprocessor from accessing internal objects and calling their methods directly until HBASE-4047. (Maybe that makes the 1.0 list?) Related, paring down the coprocessor interfaces to only intercept RPC based actions and move everything else to plugins. -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: DISCUSSION: 1.0.0
Lily's indexer uses hbase private apis in the replication portion of code. we may want to expose and make promises on the replication api/wire to make their lives easier and to enable cross version replication for future hbases. Jon. On Tue, Feb 11, 2014 at 11:21 AM, Nick Dimiduk ndimi...@gmail.com wrote: FYI, Lily's Indexer [0] also makes use of this level of integration into HBase. Others? [0]: https://github.com/NGDATA/hbase-indexer On Tue, Feb 11, 2014 at 12:09 PM, Enis Söztutar enis@gmail.com wrote: This is good discussion, I mentioned this in phoenix-dev that we need to repurpose coprocessors, and just give them put / get kind of injection points, but not allow co-processors to tap into log / compaction, etc. Those will be better served by explicitly defined plugins (like the recent StorageEngine, HLog, etc) that we explicitly allow injecting code with some API stability (though probably still evolving at a fast pace). It would be good to have an idea of what classes / methods, phoenix and similar depend on HBase right now. Enis On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong jzh...@hortonworks.com wrote: We need to revisit which interfaces should be marked as real private because coprocessors allow custom applications to accesses internal states. Current private interface such as KeyValue, WALEdit, Hregion, HLogKey, Store etc is may used by custom coprocessor applications so any public method signature change in those private interfaces possibliy breaks custom coprocessor implementations. -Jeffrey On 2/10/14 4:59 PM, lars hofhansl la...@apache.org wrote: True. But we can clearly annotate what is public and what is not. In HRegion we'd have to do it per method. It's probably easier to extract the public interface out into a Interface. (similar to Store and HStore) HBASE-4207 solves a slightly different problem. Here I want to have access to some HBase internals for performance, but I want to know what is public and stable and what I should not touch (thinking about things like Phoenix). -- Lars From: Andrew Purtell apurt...@apache.org To: dev@hbase.apache.org dev@hbase.apache.org Sent: Monday, February 10, 2014 4:46 PM Subject: Re: DISCUSSION: 1.0.0 On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl la...@apache.org wrote: + known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. We started this with Environment and the HTable wrapper as an example of how to wrap an interface for CPs. At the end of the day we can't stop a coprocessor from accessing internal objects and calling their methods directly until HBASE-4047. (Maybe that makes the 1.0 list?) Related, paring down the coprocessor interfaces to only intercept RPC based actions and move everything else to plugins. -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- // Jonathan Hsieh (shay) // HBase Tech Lead, Software Engineer, Cloudera // j...@cloudera.com // @jmhsieh
Re: DISCUSSION: 1.0.0
A few us have gotten the We need a Replication Interface defined along another channel. Agree this is a good discussion. I'll file issues to catch the above against the 0.99 label. (SighIf only we had a RM for 1.0.0...) St.Ack On Tue, Feb 11, 2014 at 11:21 AM, Nick Dimiduk ndimi...@gmail.com wrote: FYI, Lily's Indexer [0] also makes use of this level of integration into HBase. Others? [0]: https://github.com/NGDATA/hbase-indexer On Tue, Feb 11, 2014 at 12:09 PM, Enis Söztutar enis@gmail.com wrote: This is good discussion, I mentioned this in phoenix-dev that we need to repurpose coprocessors, and just give them put / get kind of injection points, but not allow co-processors to tap into log / compaction, etc. Those will be better served by explicitly defined plugins (like the recent StorageEngine, HLog, etc) that we explicitly allow injecting code with some API stability (though probably still evolving at a fast pace). It would be good to have an idea of what classes / methods, phoenix and similar depend on HBase right now. Enis On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong jzh...@hortonworks.com wrote: We need to revisit which interfaces should be marked as real private because coprocessors allow custom applications to accesses internal states. Current private interface such as KeyValue, WALEdit, Hregion, HLogKey, Store etc is may used by custom coprocessor applications so any public method signature change in those private interfaces possibliy breaks custom coprocessor implementations. -Jeffrey On 2/10/14 4:59 PM, lars hofhansl la...@apache.org wrote: True. But we can clearly annotate what is public and what is not. In HRegion we'd have to do it per method. It's probably easier to extract the public interface out into a Interface. (similar to Store and HStore) HBASE-4207 solves a slightly different problem. Here I want to have access to some HBase internals for performance, but I want to know what is public and stable and what I should not touch (thinking about things like Phoenix). -- Lars From: Andrew Purtell apurt...@apache.org To: dev@hbase.apache.org dev@hbase.apache.org Sent: Monday, February 10, 2014 4:46 PM Subject: Re: DISCUSSION: 1.0.0 On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl la...@apache.org wrote: + known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. We started this with Environment and the HTable wrapper as an example of how to wrap an interface for CPs. At the end of the day we can't stop a coprocessor from accessing internal objects and calling their methods directly until HBASE-4047. (Maybe that makes the 1.0 list?) Related, paring down the coprocessor interfaces to only intercept RPC based actions and move everything else to plugins. -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: DISCUSSION: 1.0.0
(SighIf only we had a RM for 1.0.0...) Thanks Stack for bringing this up. I actually want to volunteer as an RM for 1.0 release. We can do a 0.99 release first as a dev release, then turn 0.99.x into 1.0 as we did in 0.95 - 0.96 releases. As for timing, we can start by filing some jiras for the discussion items and setting the target versions for those to see where we are at. 0.98 will be out soon, and we want to make 1.0 sooner rather than later. Also if we can nail the release before HBaseCon that would be good with users. Enis On Tue, Feb 11, 2014 at 12:46 PM, Stack st...@duboce.net wrote: A few us have gotten the We need a Replication Interface defined along another channel. Agree this is a good discussion. I'll file issues to catch the above against the 0.99 label. (SighIf only we had a RM for 1.0.0...) St.Ack On Tue, Feb 11, 2014 at 11:21 AM, Nick Dimiduk ndimi...@gmail.com wrote: FYI, Lily's Indexer [0] also makes use of this level of integration into HBase. Others? [0]: https://github.com/NGDATA/hbase-indexer On Tue, Feb 11, 2014 at 12:09 PM, Enis Söztutar enis@gmail.com wrote: This is good discussion, I mentioned this in phoenix-dev that we need to repurpose coprocessors, and just give them put / get kind of injection points, but not allow co-processors to tap into log / compaction, etc. Those will be better served by explicitly defined plugins (like the recent StorageEngine, HLog, etc) that we explicitly allow injecting code with some API stability (though probably still evolving at a fast pace). It would be good to have an idea of what classes / methods, phoenix and similar depend on HBase right now. Enis On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong jzh...@hortonworks.com wrote: We need to revisit which interfaces should be marked as real private because coprocessors allow custom applications to accesses internal states. Current private interface such as KeyValue, WALEdit, Hregion, HLogKey, Store etc is may used by custom coprocessor applications so any public method signature change in those private interfaces possibliy breaks custom coprocessor implementations. -Jeffrey On 2/10/14 4:59 PM, lars hofhansl la...@apache.org wrote: True. But we can clearly annotate what is public and what is not. In HRegion we'd have to do it per method. It's probably easier to extract the public interface out into a Interface. (similar to Store and HStore) HBASE-4207 solves a slightly different problem. Here I want to have access to some HBase internals for performance, but I want to know what is public and stable and what I should not touch (thinking about things like Phoenix). -- Lars From: Andrew Purtell apurt...@apache.org To: dev@hbase.apache.org dev@hbase.apache.org Sent: Monday, February 10, 2014 4:46 PM Subject: Re: DISCUSSION: 1.0.0 On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl la...@apache.org wrote: + known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. We started this with Environment and the HTable wrapper as an example of how to wrap an interface for CPs. At the end of the day we can't stop a coprocessor from accessing internal objects and calling their methods directly until HBASE-4047. (Maybe that makes the 1.0 list?) Related, paring down the coprocessor interfaces to only intercept RPC based actions and move everything else to plugins. -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: DISCUSSION: 1.0.0
On Tue, Feb 11, 2014 at 1:06 PM, Enis Söztutar enis@gmail.com wrote: (SighIf only we had a RM for 1.0.0...) Thanks Stack for bringing this up. I actually want to volunteer as an RM for 1.0 release. We can do a 0.99 release first as a dev release, then turn 0.99.x into 1.0 as we did in 0.95 - 0.96 releases. As for timing, we can start by filing some jiras for the discussion items and setting the target versions for those to see where we are at. 0.98 will be out soon, and we want to make 1.0 sooner rather than later. Also if we can nail the release before HBaseCon that would be good with users. I think it'd be great if you did it Enis. It looks like the old-guard RMs are up for helping out too (going by comments above). Do we have to vote on it? Unless someone else wants to do it (speak up!), I'd think not. Yeah, would be sweet if it arrived in time for hbasecon. It'd be good to hit our 3/4 month cadence two releases in a row. 0.99 dev release sounds right too. Good stuff, St.Ack
Re: DISCUSSION: 1.0.0
I think it'd be great if you did it Enis. It looks like the old-guard RMs are up for helping out too (going by comments above). Since this is 1.0, we would need help from everybody! Do we have to vote on it? Unless someone else wants to do it (speak up!), I'd think not. Yeah, I was not sure whether we should call a vote for it. If any there are other volunteers, I can also do RM duty for a 1.1 or so release. Otherwise consider myself the official RM :) Yeah, would be sweet if it arrived in time for hbasecon. It'd be good to hit our 3/4 month cadence two releases in a row. 0.99 dev release sounds right too. Good stuff, St.Ack
Re: DISCUSSION: 1.0.0
On Tue, Feb 11, 2014 at 1:46 PM, Stack st...@duboce.net wrote: A few us have gotten the We need a Replication Interface defined along another channel. I've set HBASE-9507 to 0.99.0. Is there a better way to mark it (and these 1.0 issues in general) for further discussion? On Tue, Feb 11, 2014 at 11:21 AM, Nick Dimiduk ndimi...@gmail.com wrote: FYI, Lily's Indexer [0] also makes use of this level of integration into HBase. Others? [0]: https://github.com/NGDATA/hbase-indexer On Tue, Feb 11, 2014 at 12:09 PM, Enis Söztutar enis@gmail.com wrote: This is good discussion, I mentioned this in phoenix-dev that we need to repurpose coprocessors, and just give them put / get kind of injection points, but not allow co-processors to tap into log / compaction, etc. Those will be better served by explicitly defined plugins (like the recent StorageEngine, HLog, etc) that we explicitly allow injecting code with some API stability (though probably still evolving at a fast pace). It would be good to have an idea of what classes / methods, phoenix and similar depend on HBase right now. Enis On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong jzh...@hortonworks.com wrote: We need to revisit which interfaces should be marked as real private because coprocessors allow custom applications to accesses internal states. Current private interface such as KeyValue, WALEdit, Hregion, HLogKey, Store etc is may used by custom coprocessor applications so any public method signature change in those private interfaces possibliy breaks custom coprocessor implementations. -Jeffrey On 2/10/14 4:59 PM, lars hofhansl la...@apache.org wrote: True. But we can clearly annotate what is public and what is not. In HRegion we'd have to do it per method. It's probably easier to extract the public interface out into a Interface. (similar to Store and HStore) HBASE-4207 solves a slightly different problem. Here I want to have access to some HBase internals for performance, but I want to know what is public and stable and what I should not touch (thinking about things like Phoenix). -- Lars From: Andrew Purtell apurt...@apache.org To: dev@hbase.apache.org dev@hbase.apache.org Sent: Monday, February 10, 2014 4:46 PM Subject: Re: DISCUSSION: 1.0.0 On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl la...@apache.org wrote: + known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. We started this with Environment and the HTable wrapper as an example of how to wrap an interface for CPs. At the end of the day we can't stop a coprocessor from accessing internal objects and calling their methods directly until HBASE-4047. (Maybe that makes the 1.0 list?) Related, paring down the coprocessor interfaces to only intercept RPC based actions and move everything else to plugins. -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: DISCUSSION: 1.0.0
On Tue, Feb 11, 2014 at 2:40 PM, Enis Söztutar enis@gmail.com wrote: I think it'd be great if you did it Enis. It looks like the old-guard RMs are up for helping out too (going by comments above). Since this is 1.0, we would need help from everybody! Amen. Do we have to vote on it? Unless someone else wants to do it (speak up!), I'd think not. Yeah, I was not sure whether we should call a vote for it. If any there are other volunteers, I can also do RM duty for a 1.1 or so release. Otherwise consider myself the official RM :) Let me put it up on its own thread at least St.Ack
Re: DISCUSSION: 1.0.0
Suggestions for 1.0.0 if if it is to come out in next month or so: + Update included libs (e.g. move to log4j2) + Enable distributed log replay as default (fix bugs) + Enable hfilev3 as default. + Ship with default logging level set to INFO and content of the logs still makes sense What else? + Enable dynamic config and schema by default. St.Ack On Mon, Jan 20, 2014 at 4:09 PM, Stack st...@duboce.net wrote: On Mon, Jan 20, 2014 at 1:52 PM, lars hofhansl la...@apache.org wrote: I'm happy to volunteer. Happy if Enis does it, too. I'd be happy to do it too but my thinking is that it is good to spread the role around. St.Ack -- *From:* Stack st...@duboce.net *To:* HBase Dev List dev@hbase.apache.org *Cc:* lars hofhansl la...@apache.org *Sent:* Monday, January 20, 2014 1:43 PM *Subject:* Re: DISCUSSION: 1.0.0 On Thu, Jan 2, 2014 at 1:20 PM, Enis Söztutar e...@apache.org wrote: I think whether we will need a new RM will depend on the decision to release 1.0 from 0.98 branches or 0.99 branches(current trunk). I think it should have an RM regardless. We should probably try to put a higher polish on a 1.0 than we would mayhaps on a lesser release. RM will have enough work on their plate just keeping up state (IMO). We can do the previous practice of releasing 0.99.0, then turning 0.99.x as the 1.0.0. In that case, I can also volunteer as well. Good by me. Anyone else interested in the job? Speak up if so. If not, you'd get it by default Enis. Else you and whoever will have to dook it out. St.Ack
Re: DISCUSSION: 1.0.0
What's about reducing the default log level? 2014-02-10 12:24 GMT-05:00 Stack st...@duboce.net: Suggestions for 1.0.0 if if it is to come out in next month or so: + Update included libs (e.g. move to log4j2) + Enable distributed log replay as default (fix bugs) + Enable hfilev3 as default. + Ship with default logging level set to INFO and content of the logs still makes sense What else? + Enable dynamic config and schema by default. St.Ack On Mon, Jan 20, 2014 at 4:09 PM, Stack st...@duboce.net wrote: On Mon, Jan 20, 2014 at 1:52 PM, lars hofhansl la...@apache.org wrote: I'm happy to volunteer. Happy if Enis does it, too. I'd be happy to do it too but my thinking is that it is good to spread the role around. St.Ack -- *From:* Stack st...@duboce.net *To:* HBase Dev List dev@hbase.apache.org *Cc:* lars hofhansl la...@apache.org *Sent:* Monday, January 20, 2014 1:43 PM *Subject:* Re: DISCUSSION: 1.0.0 On Thu, Jan 2, 2014 at 1:20 PM, Enis Söztutar e...@apache.org wrote: I think whether we will need a new RM will depend on the decision to release 1.0 from 0.98 branches or 0.99 branches(current trunk). I think it should have an RM regardless. We should probably try to put a higher polish on a 1.0 than we would mayhaps on a lesser release. RM will have enough work on their plate just keeping up state (IMO). We can do the previous practice of releasing 0.99.0, then turning 0.99.x as the 1.0.0. In that case, I can also volunteer as well. Good by me. Anyone else interested in the job? Speak up if so. If not, you'd get it by default Enis. Else you and whoever will have to dook it out. St.Ack
Re: DISCUSSION: 1.0.0
On Mon, Feb 10, 2014 at 9:31 AM, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote: What's about reducing the default log level? Is the above different from '...Ship with default logging level set to INFO'? Thanks JMS, St.Ack 2014-02-10 12:24 GMT-05:00 Stack st...@duboce.net: Suggestions for 1.0.0 if if it is to come out in next month or so: + Update included libs (e.g. move to log4j2) + Enable distributed log replay as default (fix bugs) + Enable hfilev3 as default. + Ship with default logging level set to INFO and content of the logs still makes sense What else? + Enable dynamic config and schema by default. St.Ack On Mon, Jan 20, 2014 at 4:09 PM, Stack st...@duboce.net wrote: On Mon, Jan 20, 2014 at 1:52 PM, lars hofhansl la...@apache.org wrote: I'm happy to volunteer. Happy if Enis does it, too. I'd be happy to do it too but my thinking is that it is good to spread the role around. St.Ack -- *From:* Stack st...@duboce.net *To:* HBase Dev List dev@hbase.apache.org *Cc:* lars hofhansl la...@apache.org *Sent:* Monday, January 20, 2014 1:43 PM *Subject:* Re: DISCUSSION: 1.0.0 On Thu, Jan 2, 2014 at 1:20 PM, Enis Söztutar e...@apache.org wrote: I think whether we will need a new RM will depend on the decision to release 1.0 from 0.98 branches or 0.99 branches(current trunk). I think it should have an RM regardless. We should probably try to put a higher polish on a 1.0 than we would mayhaps on a lesser release. RM will have enough work on their plate just keeping up state (IMO). We can do the previous practice of releasing 0.99.0, then turning 0.99.x as the 1.0.0. In that case, I can also volunteer as well. Good by me. Anyone else interested in the job? Speak up if so. If not, you'd get it by default Enis. Else you and whoever will have to dook it out. St.Ack
Re: DISCUSSION: 1.0.0
wow. Wondering how I totally skipped this line :( Sorry. So please disregard my initial email. Thanks, JM 2014-02-10 13:52 GMT-05:00 Stack st...@duboce.net: On Mon, Feb 10, 2014 at 9:31 AM, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote: What's about reducing the default log level? Is the above different from '...Ship with default logging level set to INFO'? Thanks JMS, St.Ack 2014-02-10 12:24 GMT-05:00 Stack st...@duboce.net: Suggestions for 1.0.0 if if it is to come out in next month or so: + Update included libs (e.g. move to log4j2) + Enable distributed log replay as default (fix bugs) + Enable hfilev3 as default. + Ship with default logging level set to INFO and content of the logs still makes sense What else? + Enable dynamic config and schema by default. St.Ack On Mon, Jan 20, 2014 at 4:09 PM, Stack st...@duboce.net wrote: On Mon, Jan 20, 2014 at 1:52 PM, lars hofhansl la...@apache.org wrote: I'm happy to volunteer. Happy if Enis does it, too. I'd be happy to do it too but my thinking is that it is good to spread the role around. St.Ack -- *From:* Stack st...@duboce.net *To:* HBase Dev List dev@hbase.apache.org *Cc:* lars hofhansl la...@apache.org *Sent:* Monday, January 20, 2014 1:43 PM *Subject:* Re: DISCUSSION: 1.0.0 On Thu, Jan 2, 2014 at 1:20 PM, Enis Söztutar e...@apache.org wrote: I think whether we will need a new RM will depend on the decision to release 1.0 from 0.98 branches or 0.99 branches(current trunk). I think it should have an RM regardless. We should probably try to put a higher polish on a 1.0 than we would mayhaps on a lesser release. RM will have enough work on their plate just keeping up state (IMO). We can do the previous practice of releasing 0.99.0, then turning 0.99.x as the 1.0.0. In that case, I can also volunteer as well. Good by me. Anyone else interested in the job? Speak up if so. If not, you'd get it by default Enis. Else you and whoever will have to dook it out. St.Ack
Re: DISCUSSION: 1.0.0
On 10 February 2014 09:24, Stack st...@duboce.net wrote: Suggestions for 1.0.0 if if it is to come out in next month or so: + Update included libs (e.g. move to log4j2) love to see how that goes on -it looks good but I've not seen it in production yet. I do note it has a NoSQL outputter, so there are good opportunities for recursion + Enable distributed log replay as default (fix bugs) + Enable hfilev3 as default. + Ship with default logging level set to INFO and content of the logs still makes sense What else? + maybe make the hconnection output a bit less verbose when ZK isn't live or HBase still coming up; it does generate a fair amount of log data + light pastel purple colour scheme. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: DISCUSSION: 1.0.0
On Mon, Feb 10, 2014 at 11:15 AM, Steve Loughran ste...@hortonworks.comwrote: On 10 February 2014 09:24, Stack st...@duboce.net wrote: Suggestions for 1.0.0 if if it is to come out in next month or so: + Update included libs (e.g. move to log4j2) love to see how that goes on -it looks good but I've not seen it in production yet. I do note it has a NoSQL outputter, so there are good opportunities for recursion Smile.
Re: DISCUSSION: 1.0.0
I'd like to see a bunch of our @Deprecated APIs cleaned up. Delete stale methods or remove the annotations. On Mon, Feb 10, 2014 at 10:24 AM, Stack st...@duboce.net wrote: Suggestions for 1.0.0 if if it is to come out in next month or so: + Update included libs (e.g. move to log4j2) + Enable distributed log replay as default (fix bugs) + Enable hfilev3 as default. + Ship with default logging level set to INFO and content of the logs still makes sense What else? + Enable dynamic config and schema by default. St.Ack On Mon, Jan 20, 2014 at 4:09 PM, Stack st...@duboce.net wrote: On Mon, Jan 20, 2014 at 1:52 PM, lars hofhansl la...@apache.org wrote: I'm happy to volunteer. Happy if Enis does it, too. I'd be happy to do it too but my thinking is that it is good to spread the role around. St.Ack -- *From:* Stack st...@duboce.net *To:* HBase Dev List dev@hbase.apache.org *Cc:* lars hofhansl la...@apache.org *Sent:* Monday, January 20, 2014 1:43 PM *Subject:* Re: DISCUSSION: 1.0.0 On Thu, Jan 2, 2014 at 1:20 PM, Enis Söztutar e...@apache.org wrote: I think whether we will need a new RM will depend on the decision to release 1.0 from 0.98 branches or 0.99 branches(current trunk). I think it should have an RM regardless. We should probably try to put a higher polish on a 1.0 than we would mayhaps on a lesser release. RM will have enough work on their plate just keeping up state (IMO). We can do the previous practice of releasing 0.99.0, then turning 0.99.x as the 1.0.0. In that case, I can also volunteer as well. Good by me. Anyone else interested in the job? Speak up if so. If not, you'd get it by default Enis. Else you and whoever will have to dook it out. St.Ack
Re: DISCUSSION: 1.0.0
+ fix interface audience annotations (HBASE-10462). + Continue on limiting some broad reaching interfaces (HCM, HCI, HTable) See recent HBASE-10479 discussions) + Promote HTableInterface vs HTable, getting connections from HCM and getting tables there. On Mon, Feb 10, 2014 at 11:20 AM, Stack st...@duboce.net wrote: On Mon, Feb 10, 2014 at 11:15 AM, Steve Loughran ste...@hortonworks.com wrote: On 10 February 2014 09:24, Stack st...@duboce.net wrote: Suggestions for 1.0.0 if if it is to come out in next month or so: + Update included libs (e.g. move to log4j2) love to see how that goes on -it looks good but I've not seen it in production yet. I do note it has a NoSQL outputter, so there are good opportunities for recursion Smile.
Re: DISCUSSION: 1.0.0
Keep them coming and I'll file issues against 1.0 version. Thanks lads, St.Ack On Mon, Feb 10, 2014 at 11:38 AM, Enis Söztutar enis@gmail.com wrote: + fix interface audience annotations (HBASE-10462). + Continue on limiting some broad reaching interfaces (HCM, HCI, HTable) See recent HBASE-10479 discussions) + Promote HTableInterface vs HTable, getting connections from HCM and getting tables there. On Mon, Feb 10, 2014 at 11:20 AM, Stack st...@duboce.net wrote: On Mon, Feb 10, 2014 at 11:15 AM, Steve Loughran ste...@hortonworks.com wrote: On 10 February 2014 09:24, Stack st...@duboce.net wrote: Suggestions for 1.0.0 if if it is to come out in next month or so: + Update included libs (e.g. move to log4j2) love to see how that goes on -it looks good but I've not seen it in production yet. I do note it has a NoSQL outputter, so there are good opportunities for recursion Smile.
Re: DISCUSSION: 1.0.0
+ a clearly defined compatibility strategy between patch, minor, and major versions + known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. + (wishes for ponies and unicorns, probably not going to happen) better ops supports, such as sample puppet/chef/etc scripts From: Stack st...@duboce.net To: HBase Dev List dev@hbase.apache.org Sent: Monday, February 10, 2014 4:29 PM Subject: Re: DISCUSSION: 1.0.0 Keep them coming and I'll file issues against 1.0 version. Thanks lads, St.Ack On Mon, Feb 10, 2014 at 11:38 AM, Enis Söztutar enis@gmail.com wrote: + fix interface audience annotations (HBASE-10462). + Continue on limiting some broad reaching interfaces (HCM, HCI, HTable) See recent HBASE-10479 discussions) + Promote HTableInterface vs HTable, getting connections from HCM and getting tables there. On Mon, Feb 10, 2014 at 11:20 AM, Stack st...@duboce.net wrote: On Mon, Feb 10, 2014 at 11:15 AM, Steve Loughran ste...@hortonworks.com wrote: On 10 February 2014 09:24, Stack st...@duboce.net wrote: Suggestions for 1.0.0 if if it is to come out in next month or so: + Update included libs (e.g. move to log4j2) love to see how that goes on -it looks good but I've not seen it in production yet. I do note it has a NoSQL outputter, so there are good opportunities for recursion Smile.
Re: DISCUSSION: 1.0.0
On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl la...@apache.org wrote: + known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. We started this with Environment and the HTable wrapper as an example of how to wrap an interface for CPs. At the end of the day we can't stop a coprocessor from accessing internal objects and calling their methods directly until HBASE-4047. (Maybe that makes the 1.0 list?) Related, paring down the coprocessor interfaces to only intercept RPC based actions and move everything else to plugins. -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
Re: DISCUSSION: 1.0.0
True. But we can clearly annotate what is public and what is not. In HRegion we'd have to do it per method. It's probably easier to extract the public interface out into a Interface. (similar to Store and HStore) HBASE-4207 solves a slightly different problem. Here I want to have access to some HBase internals for performance, but I want to know what is public and stable and what I should not touch (thinking about things like Phoenix). -- Lars From: Andrew Purtell apurt...@apache.org To: dev@hbase.apache.org dev@hbase.apache.org Sent: Monday, February 10, 2014 4:46 PM Subject: Re: DISCUSSION: 1.0.0 On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl la...@apache.org wrote: + known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. We started this with Environment and the HTable wrapper as an example of how to wrap an interface for CPs. At the end of the day we can't stop a coprocessor from accessing internal objects and calling their methods directly until HBASE-4047. (Maybe that makes the 1.0 list?) Related, paring down the coprocessor interfaces to only intercept RPC based actions and move everything else to plugins. -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
Re: DISCUSSION: 1.0.0
+ HBASE-8763 Combine MVCC and SeqId Currently replication is broken on same version updates in the following scenarios: when a region move or RS get restarted in the source cluster, replicated changes may come out of order to the receiving RS. There are other cases we need to keep the ordering of puts. This JIRA can be used to fix those above issues. + Replicated Meta Regions for Distributed Region Location Lookup(I haven't cut a JIRA for it but I've been thinking this for a while) On 2/10/14 4:42 PM, lars hofhansl la...@apache.org wrote: + a clearly defined compatibility strategy between patch, minor, and major versions + known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. + (wishes for ponies and unicorns, probably not going to happen) better ops supports, such as sample puppet/chef/etc scripts From: Stack st...@duboce.net To: HBase Dev List dev@hbase.apache.org Sent: Monday, February 10, 2014 4:29 PM Subject: Re: DISCUSSION: 1.0.0 Keep them coming and I'll file issues against 1.0 version. Thanks lads, St.Ack On Mon, Feb 10, 2014 at 11:38 AM, Enis Söztutar enis@gmail.com wrote: + fix interface audience annotations (HBASE-10462). + Continue on limiting some broad reaching interfaces (HCM, HCI, HTable) See recent HBASE-10479 discussions) + Promote HTableInterface vs HTable, getting connections from HCM and getting tables there. On Mon, Feb 10, 2014 at 11:20 AM, Stack st...@duboce.net wrote: On Mon, Feb 10, 2014 at 11:15 AM, Steve Loughran ste...@hortonworks.com wrote: On 10 February 2014 09:24, Stack st...@duboce.net wrote: Suggestions for 1.0.0 if if it is to come out in next month or so: + Update included libs (e.g. move to log4j2) love to see how that goes on -it looks good but I've not seen it in production yet. I do note it has a NoSQL outputter, so there are good opportunities for recursion Smile. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: DISCUSSION: 1.0.0
It is desirable if HBASE-9203, the umbrella JIRA for secondary indexing, is given some reviews. Cheers On Mon, Feb 10, 2014 at 5:41 PM, Jeffrey Zhong jzh...@hortonworks.comwrote: + HBASE-8763 Combine MVCC and SeqId Currently replication is broken on same version updates in the following scenarios: when a region move or RS get restarted in the source cluster, replicated changes may come out of order to the receiving RS. There are other cases we need to keep the ordering of puts. This JIRA can be used to fix those above issues. + Replicated Meta Regions for Distributed Region Location Lookup(I haven't cut a JIRA for it but I've been thinking this for a while) On 2/10/14 4:42 PM, lars hofhansl la...@apache.org wrote: + a clearly defined compatibility strategy between patch, minor, and major versions + known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. + (wishes for ponies and unicorns, probably not going to happen) better ops supports, such as sample puppet/chef/etc scripts From: Stack st...@duboce.net To: HBase Dev List dev@hbase.apache.org Sent: Monday, February 10, 2014 4:29 PM Subject: Re: DISCUSSION: 1.0.0 Keep them coming and I'll file issues against 1.0 version. Thanks lads, St.Ack On Mon, Feb 10, 2014 at 11:38 AM, Enis Söztutar enis@gmail.com wrote: + fix interface audience annotations (HBASE-10462). + Continue on limiting some broad reaching interfaces (HCM, HCI, HTable) See recent HBASE-10479 discussions) + Promote HTableInterface vs HTable, getting connections from HCM and getting tables there. On Mon, Feb 10, 2014 at 11:20 AM, Stack st...@duboce.net wrote: On Mon, Feb 10, 2014 at 11:15 AM, Steve Loughran ste...@hortonworks.com wrote: On 10 February 2014 09:24, Stack st...@duboce.net wrote: Suggestions for 1.0.0 if if it is to come out in next month or so: + Update included libs (e.g. move to log4j2) love to see how that goes on -it looks good but I've not seen it in production yet. I do note it has a NoSQL outputter, so there are good opportunities for recursion Smile. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: DISCUSSION: 1.0.0
We need to revisit which interfaces should be marked as real private because coprocessors allow custom applications to accesses internal states. Current private interface such as KeyValue, WALEdit, Hregion, HLogKey, Store etc is may used by custom coprocessor applications so any public method signature change in those private interfaces possibliy breaks custom coprocessor implementations. -Jeffrey On 2/10/14 4:59 PM, lars hofhansl la...@apache.org wrote: True. But we can clearly annotate what is public and what is not. In HRegion we'd have to do it per method. It's probably easier to extract the public interface out into a Interface. (similar to Store and HStore) HBASE-4207 solves a slightly different problem. Here I want to have access to some HBase internals for performance, but I want to know what is public and stable and what I should not touch (thinking about things like Phoenix). -- Lars From: Andrew Purtell apurt...@apache.org To: dev@hbase.apache.org dev@hbase.apache.org Sent: Monday, February 10, 2014 4:46 PM Subject: Re: DISCUSSION: 1.0.0 On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl la...@apache.org wrote: + known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. We started this with Environment and the HTable wrapper as an example of how to wrap an interface for CPs. At the end of the day we can't stop a coprocessor from accessing internal objects and calling their methods directly until HBASE-4047. (Maybe that makes the 1.0 list?) Related, paring down the coprocessor interfaces to only intercept RPC based actions and move everything else to plugins. -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: DISCUSSION: 1.0.0
On Thu, Jan 2, 2014 at 1:20 PM, Enis Söztutar e...@apache.org wrote: I think whether we will need a new RM will depend on the decision to release 1.0 from 0.98 branches or 0.99 branches(current trunk). I think it should have an RM regardless. We should probably try to put a higher polish on a 1.0 than we would mayhaps on a lesser release. RM will have enough work on their plate just keeping up state (IMO). We can do the previous practice of releasing 0.99.0, then turning 0.99.x as the 1.0.0. In that case, I can also volunteer as well. Good by me. Anyone else interested in the job? Speak up if so. If not, you'd get it by default Enis. Else you and whoever will have to dook it out. St.Ack
Re: DISCUSSION: 1.0.0
I'm happy to volunteer. Happy if Enis does it, too. From: Stack st...@duboce.net To: HBase Dev List dev@hbase.apache.org Cc: lars hofhansl la...@apache.org Sent: Monday, January 20, 2014 1:43 PM Subject: Re: DISCUSSION: 1.0.0 On Thu, Jan 2, 2014 at 1:20 PM, Enis Söztutar e...@apache.org wrote: I think whether we will need a new RM will depend on the decision to release 1.0 from 0.98 branches or 0.99 branches(current trunk). I think it should have an RM regardless. We should probably try to put a higher polish on a 1.0 than we would mayhaps on a lesser release. RM will have enough work on their plate just keeping up state (IMO). We can do the previous practice of releasing 0.99.0, then turning 0.99.x as the 1.0.0. In that case, I can also volunteer as well. Good by me. Anyone else interested in the job? Speak up if so. If not, you'd get it by default Enis. Else you and whoever will have to dook it out. St.Ack
Re: DISCUSSION: 1.0.0
On Mon, Jan 20, 2014 at 1:52 PM, lars hofhansl la...@apache.org wrote: I'm happy to volunteer. Happy if Enis does it, too. I'd be happy to do it too but my thinking is that it is good to spread the role around. St.Ack -- *From:* Stack st...@duboce.net *To:* HBase Dev List dev@hbase.apache.org *Cc:* lars hofhansl la...@apache.org *Sent:* Monday, January 20, 2014 1:43 PM *Subject:* Re: DISCUSSION: 1.0.0 On Thu, Jan 2, 2014 at 1:20 PM, Enis Söztutar e...@apache.org wrote: I think whether we will need a new RM will depend on the decision to release 1.0 from 0.98 branches or 0.99 branches(current trunk). I think it should have an RM regardless. We should probably try to put a higher polish on a 1.0 than we would mayhaps on a lesser release. RM will have enough work on their plate just keeping up state (IMO). We can do the previous practice of releasing 0.99.0, then turning 0.99.x as the 1.0.0. In that case, I can also volunteer as well. Good by me. Anyone else interested in the job? Speak up if so. If not, you'd get it by default Enis. Else you and whoever will have to dook it out. St.Ack
DISCUSSION: 1.0.0
Andrew is talking of the first 0.98RC being imminent. Time to start in on the release that will follow 0.98.x. We seem to all be good with calling it 1.0.0. Speak up if you think different. (I just added a 1.0.0 version to JIRA). + What should 1.0.0 have in it beyond what is in 0.98. + Why can't 1.0.0 just be 0.98.0, or 0.98.1 altogether? + When should it come out? I'm thinking soon after 0.98. Feb/March? (Presuming 0.98 ships in Jan). + Who should RM it? (I could but perhaps others are interested). What else should we consider achieving the state of 1.0.0ness? Happy New Year all, St.Ack
Re: DISCUSSION: 1.0.0
I'd be volunteering for RM.Maybe do it together? Personally I'd like to see some heavy production usage of 0.96 or 0.98 in the community before can call the new DNA (HBase = 0.96) 1.0. -- Lars From: Stack st...@duboce.net To: HBase Dev List dev@hbase.apache.org Sent: Thursday, January 2, 2014 11:26 AM Subject: DISCUSSION: 1.0.0 Andrew is talking of the first 0.98RC being imminent. Time to start in on the release that will follow 0.98.x. We seem to all be good with calling it 1.0.0. Speak up if you think different. (I just added a 1.0.0 version to JIRA). + What should 1.0.0 have in it beyond what is in 0.98. + Why can't 1.0.0 just be 0.98.0, or 0.98.1 altogether? + When should it come out? I'm thinking soon after 0.98. Feb/March? (Presuming 0.98 ships in Jan). + Who should RM it? (I could but perhaps others are interested). What else should we consider achieving the state of 1.0.0ness? Happy New Year all, St.Ack
Re: DISCUSSION: 1.0.0
Yeah, I want to have an 0.98 RC in the test rig by week's end. So that would be artifacts for voting available next week. My suggestion is a 1.0 release 3 months or so from when 0.98.0 ships. This is arbitrary, but seems reasonable to me as enough time to get some production experience with 0.96 and/or 0.98, sort out the remaining API issues for a 1.0 caliber release, and do whatever else needs doing (tooling). Having one RM for any given release will avoid split-brain issues. :-) On Thu, Jan 2, 2014 at 11:52 AM, lars hofhansl la...@apache.org wrote: I'd be volunteering for RM.Maybe do it together? Personally I'd like to see some heavy production usage of 0.96 or 0.98 in the community before can call the new DNA (HBase = 0.96) 1.0. -- Lars From: Stack st...@duboce.net To: HBase Dev List dev@hbase.apache.org Sent: Thursday, January 2, 2014 11:26 AM Subject: DISCUSSION: 1.0.0 Andrew is talking of the first 0.98RC being imminent. Time to start in on the release that will follow 0.98.x. We seem to all be good with calling it 1.0.0. Speak up if you think different. (I just added a 1.0.0 version to JIRA). + What should 1.0.0 have in it beyond what is in 0.98. + Why can't 1.0.0 just be 0.98.0, or 0.98.1 altogether? + When should it come out? I'm thinking soon after 0.98. Feb/March? (Presuming 0.98 ships in Jan). + Who should RM it? (I could but perhaps others are interested). What else should we consider achieving the state of 1.0.0ness? Happy New Year all, St.Ack -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
Re: DISCUSSION: 1.0.0
I think whether we will need a new RM will depend on the decision to release 1.0 from 0.98 branches or 0.99 branches(current trunk). We can do the previous practice of releasing 0.99.0, then turning 0.99.x as the 1.0.0. In that case, I can also volunteer as well. Enis On Thu, Jan 2, 2014 at 12:53 PM, Andrew Purtell apurt...@apache.org wrote: Yeah, I want to have an 0.98 RC in the test rig by week's end. So that would be artifacts for voting available next week. My suggestion is a 1.0 release 3 months or so from when 0.98.0 ships. This is arbitrary, but seems reasonable to me as enough time to get some production experience with 0.96 and/or 0.98, sort out the remaining API issues for a 1.0 caliber release, and do whatever else needs doing (tooling). Having one RM for any given release will avoid split-brain issues. :-) On Thu, Jan 2, 2014 at 11:52 AM, lars hofhansl la...@apache.org wrote: I'd be volunteering for RM.Maybe do it together? Personally I'd like to see some heavy production usage of 0.96 or 0.98 in the community before can call the new DNA (HBase = 0.96) 1.0. -- Lars From: Stack st...@duboce.net To: HBase Dev List dev@hbase.apache.org Sent: Thursday, January 2, 2014 11:26 AM Subject: DISCUSSION: 1.0.0 Andrew is talking of the first 0.98RC being imminent. Time to start in on the release that will follow 0.98.x. We seem to all be good with calling it 1.0.0. Speak up if you think different. (I just added a 1.0.0 version to JIRA). + What should 1.0.0 have in it beyond what is in 0.98. + Why can't 1.0.0 just be 0.98.0, or 0.98.1 altogether? + When should it come out? I'm thinking soon after 0.98. Feb/March? (Presuming 0.98 ships in Jan). + Who should RM it? (I could but perhaps others are interested). What else should we consider achieving the state of 1.0.0ness? Happy New Year all, St.Ack -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
