Re: DISCUSSION: 1.0.0

2014-07-07 Thread Aditya
Do we want to have the C APIs part of 1.0.0 release. I had posted few
patches and a set of review request sometime last week.


On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote:

 On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com
 wrote:

  Moved ZK watcher  listener subtask out of scope HBASE-10909. Enis - with
  that, I guess HBASE-10909 can be marked in branch-1?
 

 Sounds good.


 
  HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase
  client from ZK (mostly it would be post-1.0 work).
 

 Not sure whether we can make it fully backwards compatible with 1.0
 clients. I guess we will see when the patches are done.


 
  Thanks,
  Mikhail
 
 
  2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net:
 
   On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com
 
   wrote:
  
Guys,
   
getting back to ZK abstraction work w.r.t. release 1.0 and
 thereafter,
   some
status update. So as we're getting closer to complete HBASE-10909, it
   looks
like the steps may be like this:
   
 - there are 2 subtasks out there not closed yet, one of which is
 about
   log
splitting (and Sergey S has submitted a patch for review), another is
abstraction of ZK watcher (this is what I've been working on in the
background; but after sketching the patch it seems like without being
   able
to modify the control flows and some changes in the module structure,
   it'd
be a lot of scaffolding code not really simplifying current code). So
  I'd
propose to descope abstraction of ZK watcher jira (HBASE-11073),
  namely:
convert it to top-level JIRA and continue to work on it separately;
   rename
HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as
 soon
   as
log splitting jira is completed. This way HBASE-10909 fits to
 branch-1.
   
  
   Sounds good to me.
  
  
 - secondly, in the discussion to the CatalogTracker patch, we
 started
talking about modifying client to not know about ZK, but rather keep
  the
location of current masters and talk to them using RPC calls. This
 work
   can
not go into branch-1, as it involves invasive changes in client
  including
new RPC. As I understand the branching schema now, those changes can
 go
   to
master branch, we just don't merge them to branch-1, and depending on
   their
completeness we can pull them to 1.1 release or so.
   
  
   You have it right Mikhail.
  
   St.Ack
  
 
 
 
  --
  Thanks,
  Michael Antonov
 



Re: DISCUSSION: 1.0.0

2014-07-07 Thread Andrew Purtell
Hi Aditya,

I think it would be good to have a native client API in 1.0. I can look at the 
patches when back in the office later this week. 


 On Jul 7, 2014, at 9:52 AM, Aditya adityakish...@gmail.com wrote:
 
 Do we want to have the C APIs part of 1.0.0 release. I had posted few
 patches and a set of review request sometime last week.
 
 
 On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote:
 
 On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com
 wrote:
 
 Moved ZK watcher  listener subtask out of scope HBASE-10909. Enis - with
 that, I guess HBASE-10909 can be marked in branch-1?
 
 Sounds good.
 
 
 
 HBASE-11464 - this is the jira where I'll capture tasks to abstract hbase
 client from ZK (mostly it would be post-1.0 work).
 
 Not sure whether we can make it fully backwards compatible with 1.0
 clients. I guess we will see when the patches are done.
 
 
 
 Thanks,
 Mikhail
 
 
 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net:
 
 On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov olorinb...@gmail.com
 
 wrote:
 
 Guys,
 
 getting back to ZK abstraction work w.r.t. release 1.0 and
 thereafter,
 some
 status update. So as we're getting closer to complete HBASE-10909, it
 looks
 like the steps may be like this:
 
 - there are 2 subtasks out there not closed yet, one of which is
 about
 log
 splitting (and Sergey S has submitted a patch for review), another is
 abstraction of ZK watcher (this is what I've been working on in the
 background; but after sketching the patch it seems like without being
 able
 to modify the control flows and some changes in the module structure,
 it'd
 be a lot of scaffolding code not really simplifying current code). So
 I'd
 propose to descope abstraction of ZK watcher jira (HBASE-11073),
 namely:
 convert it to top-level JIRA and continue to work on it separately;
 rename
 HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as
 soon
 as
 log splitting jira is completed. This way HBASE-10909 fits to
 branch-1.
 
 Sounds good to me.
 
 
 - secondly, in the discussion to the CatalogTracker patch, we
 started
 talking about modifying client to not know about ZK, but rather keep
 the
 location of current masters and talk to them using RPC calls. This
 work
 can
 not go into branch-1, as it involves invasive changes in client
 including
 new RPC. As I understand the branching schema now, those changes can
 go
 to
 master branch, we just don't merge them to branch-1, and depending on
 their
 completeness we can pull them to 1.1 release or so.
 
 You have it right Mikhail.
 
 St.Ack
 
 
 
 --
 Thanks,
 Michael Antonov
 


Re: DISCUSSION: 1.0.0

2014-07-07 Thread Nick Dimiduk
Would you mind including the JIRA numbers along with the request?

Thanks,
Nick


On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote:

 Do we want to have the C APIs part of 1.0.0 release. I had posted few
 patches and a set of review request sometime last week.


 On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote:

  On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com
  wrote:
 
   Moved ZK watcher  listener subtask out of scope HBASE-10909. Enis -
 with
   that, I guess HBASE-10909 can be marked in branch-1?
  
 
  Sounds good.
 
 
  
   HBASE-11464 - this is the jira where I'll capture tasks to abstract
 hbase
   client from ZK (mostly it would be post-1.0 work).
  
 
  Not sure whether we can make it fully backwards compatible with 1.0
  clients. I guess we will see when the patches are done.
 
 
  
   Thanks,
   Mikhail
  
  
   2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net:
  
On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov 
 olorinb...@gmail.com
  
wrote:
   
 Guys,

 getting back to ZK abstraction work w.r.t. release 1.0 and
  thereafter,
some
 status update. So as we're getting closer to complete HBASE-10909,
 it
looks
 like the steps may be like this:

  - there are 2 subtasks out there not closed yet, one of which is
  about
log
 splitting (and Sergey S has submitted a patch for review), another
 is
 abstraction of ZK watcher (this is what I've been working on in the
 background; but after sketching the patch it seems like without
 being
able
 to modify the control flows and some changes in the module
 structure,
it'd
 be a lot of scaffolding code not really simplifying current code).
 So
   I'd
 propose to descope abstraction of ZK watcher jira (HBASE-11073),
   namely:
 convert it to top-level JIRA and continue to work on it separately;
rename
 HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as
  soon
as
 log splitting jira is completed. This way HBASE-10909 fits to
  branch-1.

   
Sounds good to me.
   
   
  - secondly, in the discussion to the CatalogTracker patch, we
  started
 talking about modifying client to not know about ZK, but rather
 keep
   the
 location of current masters and talk to them using RPC calls. This
  work
can
 not go into branch-1, as it involves invasive changes in client
   including
 new RPC. As I understand the branching schema now, those changes
 can
  go
to
 master branch, we just don't merge them to branch-1, and depending
 on
their
 completeness we can pull them to 1.1 release or so.

   
You have it right Mikhail.
   
St.Ack
   
  
  
  
   --
   Thanks,
   Michael Antonov
  
 



Re: DISCUSSION: 1.0.0

2014-07-07 Thread Ted Yu
The following is another candidate for 1.0.0 release

HBASE-11447 Proposal for a generic transaction API for HBase

Cheers


On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote:

 Do we want to have the C APIs part of 1.0.0 release. I had posted few
 patches and a set of review request sometime last week.


 On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote:

  On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com
  wrote:
 
   Moved ZK watcher  listener subtask out of scope HBASE-10909. Enis -
 with
   that, I guess HBASE-10909 can be marked in branch-1?
  
 
  Sounds good.
 
 
  
   HBASE-11464 - this is the jira where I'll capture tasks to abstract
 hbase
   client from ZK (mostly it would be post-1.0 work).
  
 
  Not sure whether we can make it fully backwards compatible with 1.0
  clients. I guess we will see when the patches are done.
 
 
  
   Thanks,
   Mikhail
  
  
   2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net:
  
On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov 
 olorinb...@gmail.com
  
wrote:
   
 Guys,

 getting back to ZK abstraction work w.r.t. release 1.0 and
  thereafter,
some
 status update. So as we're getting closer to complete HBASE-10909,
 it
looks
 like the steps may be like this:

  - there are 2 subtasks out there not closed yet, one of which is
  about
log
 splitting (and Sergey S has submitted a patch for review), another
 is
 abstraction of ZK watcher (this is what I've been working on in the
 background; but after sketching the patch it seems like without
 being
able
 to modify the control flows and some changes in the module
 structure,
it'd
 be a lot of scaffolding code not really simplifying current code).
 So
   I'd
 propose to descope abstraction of ZK watcher jira (HBASE-11073),
   namely:
 convert it to top-level JIRA and continue to work on it separately;
rename
 HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as
  soon
as
 log splitting jira is completed. This way HBASE-10909 fits to
  branch-1.

   
Sounds good to me.
   
   
  - secondly, in the discussion to the CatalogTracker patch, we
  started
 talking about modifying client to not know about ZK, but rather
 keep
   the
 location of current masters and talk to them using RPC calls. This
  work
can
 not go into branch-1, as it involves invasive changes in client
   including
 new RPC. As I understand the branching schema now, those changes
 can
  go
to
 master branch, we just don't merge them to branch-1, and depending
 on
their
 completeness we can pull them to 1.1 release or so.

   
You have it right Mikhail.
   
St.Ack
   
  
  
  
   --
   Thanks,
   Michael Antonov
  
 



Re: DISCUSSION: 1.0.0

2014-07-07 Thread Mikhail Antonov
I thought the scope of HBASE-11447 is to review and refine proposed design,
not to track the implementation of the feature?

-Mikhail


2014-07-07 10:40 GMT-07:00 Andrew Purtell andrew.purt...@gmail.com:

 That is not a realistic proposal for 1.0 as far as I can see. There is
 only a tentative design spec on that issue. I assume we are sticking with
 Enis' earlier stated intent to get 1.0 out soon.


  On Jul 7, 2014, at 10:30 AM, Ted Yu yuzhih...@gmail.com wrote:
 
  The following is another candidate for 1.0.0 release
 
  HBASE-11447 Proposal for a generic transaction API for HBase
 
  Cheers
 
 
  On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote:
 
  Do we want to have the C APIs part of 1.0.0 release. I had posted few
  patches and a set of review request sometime last week.
 
 
  On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com
 wrote:
 
  On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com
  wrote:
 
  Moved ZK watcher  listener subtask out of scope HBASE-10909. Enis -
  with
  that, I guess HBASE-10909 can be marked in branch-1?
 
  Sounds good.
 
 
 
  HBASE-11464 - this is the jira where I'll capture tasks to abstract
  hbase
  client from ZK (mostly it would be post-1.0 work).
 
  Not sure whether we can make it fully backwards compatible with 1.0
  clients. I guess we will see when the patches are done.
 
 
 
  Thanks,
  Mikhail
 
 
  2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net:
 
  On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov 
  olorinb...@gmail.com
 
  wrote:
 
  Guys,
 
  getting back to ZK abstraction work w.r.t. release 1.0 and
  thereafter,
  some
  status update. So as we're getting closer to complete HBASE-10909,
  it
  looks
  like the steps may be like this:
 
  - there are 2 subtasks out there not closed yet, one of which is
  about
  log
  splitting (and Sergey S has submitted a patch for review), another
  is
  abstraction of ZK watcher (this is what I've been working on in the
  background; but after sketching the patch it seems like without
  being
  able
  to modify the control flows and some changes in the module
  structure,
  it'd
  be a lot of scaffolding code not really simplifying current code).
  So
  I'd
  propose to descope abstraction of ZK watcher jira (HBASE-11073),
  namely:
  convert it to top-level JIRA and continue to work on it separately;
  rename
  HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as
  soon
  as
  log splitting jira is completed. This way HBASE-10909 fits to
  branch-1.
 
  Sounds good to me.
 
 
  - secondly, in the discussion to the CatalogTracker patch, we
  started
  talking about modifying client to not know about ZK, but rather
  keep
  the
  location of current masters and talk to them using RPC calls. This
  work
  can
  not go into branch-1, as it involves invasive changes in client
  including
  new RPC. As I understand the branching schema now, those changes
  can
  go
  to
  master branch, we just don't merge them to branch-1, and depending
  on
  their
  completeness we can pull them to 1.1 release or so.
 
  You have it right Mikhail.
 
  St.Ack
 
 
 
  --
  Thanks,
  Michael Antonov
 




-- 
Thanks,
Michael Antonov


Re: DISCUSSION: 1.0.0

2014-07-07 Thread Enis Söztutar
I think it is fair to say for both C API and transaction API that it should
be ok to include them if they are ready before the time to cut the 1.0 RC.
If not, I do not want to hold the release waiting for those.

Enis


On Mon, Jul 7, 2014 at 10:50 AM, Andrew Purtell andrew.purt...@gmail.com
wrote:

 And by definition of that being a proposed design, there is no
 implementation to track (yet). There being no implementation, it's hard to
 see how it makes a 1.0 happening soon.

 Don't misunderstand me to be opposed to the idea or feature, far from it.
 I don't think Ted did anyone a service bringing it up in the context of
 1.0. Already we are on some kind of detour from discussing things where
 there are patches available.


  On Jul 7, 2014, at 10:43 AM, Mikhail Antonov olorinb...@gmail.com
 wrote:
 
  I thought the scope of HBASE-11447 is to review and refine proposed
 design,
  not to track the implementation of the feature?
 
  -Mikhail
 
 
  2014-07-07 10:40 GMT-07:00 Andrew Purtell andrew.purt...@gmail.com:
 
  That is not a realistic proposal for 1.0 as far as I can see. There is
  only a tentative design spec on that issue. I assume we are sticking
 with
  Enis' earlier stated intent to get 1.0 out soon.
 
 
  On Jul 7, 2014, at 10:30 AM, Ted Yu yuzhih...@gmail.com wrote:
 
  The following is another candidate for 1.0.0 release
 
  HBASE-11447 Proposal for a generic transaction API for HBase
 
  Cheers
 
 
  On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com
 wrote:
 
  Do we want to have the C APIs part of 1.0.0 release. I had posted few
  patches and a set of review request sometime last week.
 
 
  On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com
  wrote:
 
  On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov 
 olorinb...@gmail.com
  wrote:
 
  Moved ZK watcher  listener subtask out of scope HBASE-10909. Enis -
  with
  that, I guess HBASE-10909 can be marked in branch-1?
 
  Sounds good.
 
 
 
  HBASE-11464 - this is the jira where I'll capture tasks to abstract
  hbase
  client from ZK (mostly it would be post-1.0 work).
 
  Not sure whether we can make it fully backwards compatible with 1.0
  clients. I guess we will see when the patches are done.
 
 
 
  Thanks,
  Mikhail
 
 
  2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net:
 
  On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov 
  olorinb...@gmail.com
 
  wrote:
 
  Guys,
 
  getting back to ZK abstraction work w.r.t. release 1.0 and
  thereafter,
  some
  status update. So as we're getting closer to complete HBASE-10909,
  it
  looks
  like the steps may be like this:
 
  - there are 2 subtasks out there not closed yet, one of which is
  about
  log
  splitting (and Sergey S has submitted a patch for review), another
  is
  abstraction of ZK watcher (this is what I've been working on in
 the
  background; but after sketching the patch it seems like without
  being
  able
  to modify the control flows and some changes in the module
  structure,
  it'd
  be a lot of scaffolding code not really simplifying current code).
  So
  I'd
  propose to descope abstraction of ZK watcher jira (HBASE-11073),
  namely:
  convert it to top-level JIRA and continue to work on it
 separately;
  rename
  HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as
  soon
  as
  log splitting jira is completed. This way HBASE-10909 fits to
  branch-1.
 
  Sounds good to me.
 
 
  - secondly, in the discussion to the CatalogTracker patch, we
  started
  talking about modifying client to not know about ZK, but rather
  keep
  the
  location of current masters and talk to them using RPC calls. This
  work
  can
  not go into branch-1, as it involves invasive changes in client
  including
  new RPC. As I understand the branching schema now, those changes
  can
  go
  to
  master branch, we just don't merge them to branch-1, and depending
  on
  their
  completeness we can pull them to 1.1 release or so.
 
  You have it right Mikhail.
 
  St.Ack
 
 
 
  --
  Thanks,
  Michael Antonov
 
 
 
  --
  Thanks,
  Michael Antonov



Re: DISCUSSION: 1.0.0

2014-07-07 Thread Andrew Purtell
And by definition of that being a proposed design, there is no implementation 
to track (yet). There being no implementation, it's hard to see how it makes a 
1.0 happening soon. 

Don't misunderstand me to be opposed to the idea or feature, far from it. I 
don't think Ted did anyone a service bringing it up in the context of 1.0. 
Already we are on some kind of detour from discussing things where there are 
patches available. 


 On Jul 7, 2014, at 10:43 AM, Mikhail Antonov olorinb...@gmail.com wrote:
 
 I thought the scope of HBASE-11447 is to review and refine proposed design,
 not to track the implementation of the feature?
 
 -Mikhail
 
 
 2014-07-07 10:40 GMT-07:00 Andrew Purtell andrew.purt...@gmail.com:
 
 That is not a realistic proposal for 1.0 as far as I can see. There is
 only a tentative design spec on that issue. I assume we are sticking with
 Enis' earlier stated intent to get 1.0 out soon.
 
 
 On Jul 7, 2014, at 10:30 AM, Ted Yu yuzhih...@gmail.com wrote:
 
 The following is another candidate for 1.0.0 release
 
 HBASE-11447 Proposal for a generic transaction API for HBase
 
 Cheers
 
 
 On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote:
 
 Do we want to have the C APIs part of 1.0.0 release. I had posted few
 patches and a set of review request sometime last week.
 
 
 On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com
 wrote:
 
 On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com
 wrote:
 
 Moved ZK watcher  listener subtask out of scope HBASE-10909. Enis -
 with
 that, I guess HBASE-10909 can be marked in branch-1?
 
 Sounds good.
 
 
 
 HBASE-11464 - this is the jira where I'll capture tasks to abstract
 hbase
 client from ZK (mostly it would be post-1.0 work).
 
 Not sure whether we can make it fully backwards compatible with 1.0
 clients. I guess we will see when the patches are done.
 
 
 
 Thanks,
 Mikhail
 
 
 2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net:
 
 On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov 
 olorinb...@gmail.com
 
 wrote:
 
 Guys,
 
 getting back to ZK abstraction work w.r.t. release 1.0 and
 thereafter,
 some
 status update. So as we're getting closer to complete HBASE-10909,
 it
 looks
 like the steps may be like this:
 
 - there are 2 subtasks out there not closed yet, one of which is
 about
 log
 splitting (and Sergey S has submitted a patch for review), another
 is
 abstraction of ZK watcher (this is what I've been working on in the
 background; but after sketching the patch it seems like without
 being
 able
 to modify the control flows and some changes in the module
 structure,
 it'd
 be a lot of scaffolding code not really simplifying current code).
 So
 I'd
 propose to descope abstraction of ZK watcher jira (HBASE-11073),
 namely:
 convert it to top-level JIRA and continue to work on it separately;
 rename
 HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as
 soon
 as
 log splitting jira is completed. This way HBASE-10909 fits to
 branch-1.
 
 Sounds good to me.
 
 
 - secondly, in the discussion to the CatalogTracker patch, we
 started
 talking about modifying client to not know about ZK, but rather
 keep
 the
 location of current masters and talk to them using RPC calls. This
 work
 can
 not go into branch-1, as it involves invasive changes in client
 including
 new RPC. As I understand the branching schema now, those changes
 can
 go
 to
 master branch, we just don't merge them to branch-1, and depending
 on
 their
 completeness we can pull them to 1.1 release or so.
 
 You have it right Mikhail.
 
 St.Ack
 
 
 
 --
 Thanks,
 Michael Antonov
 
 
 
 -- 
 Thanks,
 Michael Antonov


[jira] [Created] (HBASE-11466) HConnectionImplementation should not use ZK

2014-07-07 Thread Mikhail Antonov (JIRA)
Mikhail Antonov created HBASE-11466:
---

 Summary: HConnectionImplementation should not use ZK
 Key: HBASE-11466
 URL: https://issues.apache.org/jira/browse/HBASE-11466
 Project: HBase
  Issue Type: Sub-task
  Components: Client
Affects Versions: 2.0.0
Reporter: Mikhail Antonov
 Fix For: 2.0.0


Currently ConnectionManager.HConnectionImplementation uses ZK to get address of 
current master. Should instead use pluggable interface to get location of 
master to connect to (current active master in the cluster, until we have 
multiple active masters) elsewhere (e.g. round-robin over the list of masters 
set in the client's hbase-site.xml)



--
This message was sent by Atlassian JIRA
(v6.2#6252)


Re: DISCUSSION: 1.0.0

2014-07-07 Thread Aditya
Sorry about that.

Here is the umbrella JIRA https://issues.apache.org/jira/browse/HBASE-1015


On Mon, Jul 7, 2014 at 10:05 AM, Nick Dimiduk ndimi...@gmail.com wrote:

 Would you mind including the JIRA numbers along with the request?

 Thanks,
 Nick


 On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com wrote:

 Do we want to have the C APIs part of 1.0.0 release. I had posted few
 patches and a set of review request sometime last week.


 On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com wrote:

  On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov olorinb...@gmail.com
  wrote:
 
   Moved ZK watcher  listener subtask out of scope HBASE-10909. Enis -
 with
   that, I guess HBASE-10909 can be marked in branch-1?
  
 
  Sounds good.
 
 
  
   HBASE-11464 - this is the jira where I'll capture tasks to abstract
 hbase
   client from ZK (mostly it would be post-1.0 work).
  
 
  Not sure whether we can make it fully backwards compatible with 1.0
  clients. I guess we will see when the patches are done.
 
 
  
   Thanks,
   Mikhail
  
  
   2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net:
  
On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov 
 olorinb...@gmail.com
  
wrote:
   
 Guys,

 getting back to ZK abstraction work w.r.t. release 1.0 and
  thereafter,
some
 status update. So as we're getting closer to complete
 HBASE-10909, it
looks
 like the steps may be like this:

  - there are 2 subtasks out there not closed yet, one of which is
  about
log
 splitting (and Sergey S has submitted a patch for review),
 another is
 abstraction of ZK watcher (this is what I've been working on in
 the
 background; but after sketching the patch it seems like without
 being
able
 to modify the control flows and some changes in the module
 structure,
it'd
 be a lot of scaffolding code not really simplifying current
 code). So
   I'd
 propose to descope abstraction of ZK watcher jira (HBASE-11073),
   namely:
 convert it to top-level JIRA and continue to work on it
 separately;
rename
 HBASE-10909 to ZK abstraction: phase 1, and mark it as closed as
  soon
as
 log splitting jira is completed. This way HBASE-10909 fits to
  branch-1.

   
Sounds good to me.
   
   
  - secondly, in the discussion to the CatalogTracker patch, we
  started
 talking about modifying client to not know about ZK, but rather
 keep
   the
 location of current masters and talk to them using RPC calls. This
  work
can
 not go into branch-1, as it involves invasive changes in client
   including
 new RPC. As I understand the branching schema now, those changes
 can
  go
to
 master branch, we just don't merge them to branch-1, and
 depending on
their
 completeness we can pull them to 1.1 release or so.

   
You have it right Mikhail.
   
St.Ack
   
  
  
  
   --
   Thanks,
   Michael Antonov
  
 





[jira] [Created] (HBASE-11467) New impl of Registry interface not using ZK + new RPCs on master protocol.

2014-07-07 Thread Mikhail Antonov (JIRA)
Mikhail Antonov created HBASE-11467:
---

 Summary: New impl of Registry interface not using ZK + new RPCs on 
master protocol.
 Key: HBASE-11467
 URL: https://issues.apache.org/jira/browse/HBASE-11467
 Project: HBase
  Issue Type: Sub-task
Reporter: Mikhail Antonov


Currently there' only one implementation of Registry interface, which is using 
ZK to get info about meta. Need to create implementation which will be using  
RPC calls to master the client is connected to.

That includes adding several new methods to master RPC protocol:

- GetMetaRegionLocation
- GetClusterId
- IsTableOnlineState
- GetCurrentNrHRS 



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (HBASE-11468) MasterAddressTracker needs to be moved to hbase-server

2014-07-07 Thread Mikhail Antonov (JIRA)
Mikhail Antonov created HBASE-11468:
---

 Summary: MasterAddressTracker needs to be moved to hbase-server
 Key: HBASE-11468
 URL: https://issues.apache.org/jira/browse/HBASE-11468
 Project: HBase
  Issue Type: Sub-task
Reporter: Mikhail Antonov


Later we should get rid of it. For now, for the purpose to abstract client from 
ZK, we don't use in in hbase-client (where we read master location elsewhere), 
but on the server side we can use it for now, and still keep current znode 
tracking location of current active master.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (HBASE-11469) MetaTableLocator should be moved to hbase-server

2014-07-07 Thread Mikhail Antonov (JIRA)
Mikhail Antonov created HBASE-11469:
---

 Summary: MetaTableLocator should be moved to hbase-server
 Key: HBASE-11469
 URL: https://issues.apache.org/jira/browse/HBASE-11469
 Project: HBase
  Issue Type: Sub-task
Reporter: Mikhail Antonov


It shall not be used on client side, clients shall make rpc calls to master to 
find out about meta. Also it needs to either be removed from Server interface 
available on client side and moved into RegionServer class or something, or 
whole Server interface needs to be moved to hbase-server.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (HBASE-11470) Move Server interface to hbase-server

2014-07-07 Thread Mikhail Antonov (JIRA)
Mikhail Antonov created HBASE-11470:
---

 Summary: Move Server interface to hbase-server
 Key: HBASE-11470
 URL: https://issues.apache.org/jira/browse/HBASE-11470
 Project: HBase
  Issue Type: Sub-task
Reporter: Mikhail Antonov


Looks like it's not being used on client side anyway.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


Re: DISCUSSION: 1.0.0

2014-07-07 Thread Ted Yu
We're on the same page, Enis.

Cheers


On Mon, Jul 7, 2014 at 10:56 AM, Enis Söztutar e...@apache.org wrote:

 I think it is fair to say for both C API and transaction API that it should
 be ok to include them if they are ready before the time to cut the 1.0 RC.
 If not, I do not want to hold the release waiting for those.

 Enis


 On Mon, Jul 7, 2014 at 10:50 AM, Andrew Purtell andrew.purt...@gmail.com
 wrote:

  And by definition of that being a proposed design, there is no
  implementation to track (yet). There being no implementation, it's hard
 to
  see how it makes a 1.0 happening soon.
 
  Don't misunderstand me to be opposed to the idea or feature, far from it.
  I don't think Ted did anyone a service bringing it up in the context of
  1.0. Already we are on some kind of detour from discussing things where
  there are patches available.
 
 
   On Jul 7, 2014, at 10:43 AM, Mikhail Antonov olorinb...@gmail.com
  wrote:
  
   I thought the scope of HBASE-11447 is to review and refine proposed
  design,
   not to track the implementation of the feature?
  
   -Mikhail
  
  
   2014-07-07 10:40 GMT-07:00 Andrew Purtell andrew.purt...@gmail.com:
  
   That is not a realistic proposal for 1.0 as far as I can see. There is
   only a tentative design spec on that issue. I assume we are sticking
  with
   Enis' earlier stated intent to get 1.0 out soon.
  
  
   On Jul 7, 2014, at 10:30 AM, Ted Yu yuzhih...@gmail.com wrote:
  
   The following is another candidate for 1.0.0 release
  
   HBASE-11447 Proposal for a generic transaction API for HBase
  
   Cheers
  
  
   On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com
  wrote:
  
   Do we want to have the C APIs part of 1.0.0 release. I had posted
 few
   patches and a set of review request sometime last week.
  
  
   On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com
   wrote:
  
   On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov 
  olorinb...@gmail.com
   wrote:
  
   Moved ZK watcher  listener subtask out of scope HBASE-10909.
 Enis -
   with
   that, I guess HBASE-10909 can be marked in branch-1?
  
   Sounds good.
  
  
  
   HBASE-11464 - this is the jira where I'll capture tasks to
 abstract
   hbase
   client from ZK (mostly it would be post-1.0 work).
  
   Not sure whether we can make it fully backwards compatible with 1.0
   clients. I guess we will see when the patches are done.
  
  
  
   Thanks,
   Mikhail
  
  
   2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net:
  
   On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov 
   olorinb...@gmail.com
  
   wrote:
  
   Guys,
  
   getting back to ZK abstraction work w.r.t. release 1.0 and
   thereafter,
   some
   status update. So as we're getting closer to complete
 HBASE-10909,
   it
   looks
   like the steps may be like this:
  
   - there are 2 subtasks out there not closed yet, one of which is
   about
   log
   splitting (and Sergey S has submitted a patch for review),
 another
   is
   abstraction of ZK watcher (this is what I've been working on in
  the
   background; but after sketching the patch it seems like without
   being
   able
   to modify the control flows and some changes in the module
   structure,
   it'd
   be a lot of scaffolding code not really simplifying current
 code).
   So
   I'd
   propose to descope abstraction of ZK watcher jira (HBASE-11073),
   namely:
   convert it to top-level JIRA and continue to work on it
  separately;
   rename
   HBASE-10909 to ZK abstraction: phase 1, and mark it as closed
 as
   soon
   as
   log splitting jira is completed. This way HBASE-10909 fits to
   branch-1.
  
   Sounds good to me.
  
  
   - secondly, in the discussion to the CatalogTracker patch, we
   started
   talking about modifying client to not know about ZK, but rather
   keep
   the
   location of current masters and talk to them using RPC calls.
 This
   work
   can
   not go into branch-1, as it involves invasive changes in client
   including
   new RPC. As I understand the branching schema now, those changes
   can
   go
   to
   master branch, we just don't merge them to branch-1, and
 depending
   on
   their
   completeness we can pull them to 1.1 release or so.
  
   You have it right Mikhail.
  
   St.Ack
  
  
  
   --
   Thanks,
   Michael Antonov
  
  
  
   --
   Thanks,
   Michael Antonov
 



[jira] [Created] (HBASE-11471) Move TableStateManager and ZkTableStateManager to hbase-server

2014-07-07 Thread Mikhail Antonov (JIRA)
Mikhail Antonov created HBASE-11471:
---

 Summary: Move TableStateManager and ZkTableStateManager to 
hbase-server
 Key: HBASE-11471
 URL: https://issues.apache.org/jira/browse/HBASE-11471
 Project: HBase
  Issue Type: Sub-task
Reporter: Mikhail Antonov






--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (HBASE-11472) Remove ZKTableStateClientSideReader class

2014-07-07 Thread Mikhail Antonov (JIRA)
Mikhail Antonov created HBASE-11472:
---

 Summary: Remove ZKTableStateClientSideReader class
 Key: HBASE-11472
 URL: https://issues.apache.org/jira/browse/HBASE-11472
 Project: HBase
  Issue Type: Sub-task
Reporter: Mikhail Antonov


On client side this is now only used by ZooKeeperRegistry. Shouldn't be needed 
once we have registry implementation not using ZK. On server side we should be 
using TableStateManager instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (HBASE-11473) Add BaseWALObserver class

2014-07-07 Thread Enis Soztutar (JIRA)
Enis Soztutar created HBASE-11473:
-

 Summary: Add BaseWALObserver class
 Key: HBASE-11473
 URL: https://issues.apache.org/jira/browse/HBASE-11473
 Project: HBase
  Issue Type: Improvement
Reporter: Enis Soztutar
Assignee: Enis Soztutar
 Fix For: 0.99.0, 0.98.4, 2.0.0


We lack a BaseWALObserver class for WALObserver coprocessors. Other coprocessor 
interfaces come with a default base class which makes life a little bit easier 
for coprocessor implementors. 

I think we can add this to 0.98+. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)


Re: DISCUSSION: 1.0.0

2014-07-07 Thread Jonathan Hsieh
I agree.  These are fine things we could get in  1.1 or 1.2 release if they
don't break compat and otherwise a 2.0 release timeframe.

Jon.


On Mon, Jul 7, 2014 at 10:56 AM, Enis Söztutar e...@apache.org wrote:

 I think it is fair to say for both C API and transaction API that it should
 be ok to include them if they are ready before the time to cut the 1.0 RC.
 If not, I do not want to hold the release waiting for those.

 Enis


 On Mon, Jul 7, 2014 at 10:50 AM, Andrew Purtell andrew.purt...@gmail.com
 wrote:

  And by definition of that being a proposed design, there is no
  implementation to track (yet). There being no implementation, it's hard
 to
  see how it makes a 1.0 happening soon.
 
  Don't misunderstand me to be opposed to the idea or feature, far from it.
  I don't think Ted did anyone a service bringing it up in the context of
  1.0. Already we are on some kind of detour from discussing things where
  there are patches available.
 
 
   On Jul 7, 2014, at 10:43 AM, Mikhail Antonov olorinb...@gmail.com
  wrote:
  
   I thought the scope of HBASE-11447 is to review and refine proposed
  design,
   not to track the implementation of the feature?
  
   -Mikhail
  
  
   2014-07-07 10:40 GMT-07:00 Andrew Purtell andrew.purt...@gmail.com:
  
   That is not a realistic proposal for 1.0 as far as I can see. There is
   only a tentative design spec on that issue. I assume we are sticking
  with
   Enis' earlier stated intent to get 1.0 out soon.
  
  
   On Jul 7, 2014, at 10:30 AM, Ted Yu yuzhih...@gmail.com wrote:
  
   The following is another candidate for 1.0.0 release
  
   HBASE-11447 Proposal for a generic transaction API for HBase
  
   Cheers
  
  
   On Mon, Jul 7, 2014 at 9:52 AM, Aditya adityakish...@gmail.com
  wrote:
  
   Do we want to have the C APIs part of 1.0.0 release. I had posted
 few
   patches and a set of review request sometime last week.
  
  
   On Fri, Jul 4, 2014 at 1:21 AM, Enis Söztutar enis@gmail.com
   wrote:
  
   On Thu, Jul 3, 2014 at 4:41 PM, Mikhail Antonov 
  olorinb...@gmail.com
   wrote:
  
   Moved ZK watcher  listener subtask out of scope HBASE-10909.
 Enis -
   with
   that, I guess HBASE-10909 can be marked in branch-1?
  
   Sounds good.
  
  
  
   HBASE-11464 - this is the jira where I'll capture tasks to
 abstract
   hbase
   client from ZK (mostly it would be post-1.0 work).
  
   Not sure whether we can make it fully backwards compatible with 1.0
   clients. I guess we will see when the patches are done.
  
  
  
   Thanks,
   Mikhail
  
  
   2014-07-03 12:52 GMT-07:00 Stack st...@duboce.net:
  
   On Thu, Jul 3, 2014 at 12:25 PM, Mikhail Antonov 
   olorinb...@gmail.com
  
   wrote:
  
   Guys,
  
   getting back to ZK abstraction work w.r.t. release 1.0 and
   thereafter,
   some
   status update. So as we're getting closer to complete
 HBASE-10909,
   it
   looks
   like the steps may be like this:
  
   - there are 2 subtasks out there not closed yet, one of which is
   about
   log
   splitting (and Sergey S has submitted a patch for review),
 another
   is
   abstraction of ZK watcher (this is what I've been working on in
  the
   background; but after sketching the patch it seems like without
   being
   able
   to modify the control flows and some changes in the module
   structure,
   it'd
   be a lot of scaffolding code not really simplifying current
 code).
   So
   I'd
   propose to descope abstraction of ZK watcher jira (HBASE-11073),
   namely:
   convert it to top-level JIRA and continue to work on it
  separately;
   rename
   HBASE-10909 to ZK abstraction: phase 1, and mark it as closed
 as
   soon
   as
   log splitting jira is completed. This way HBASE-10909 fits to
   branch-1.
  
   Sounds good to me.
  
  
   - secondly, in the discussion to the CatalogTracker patch, we
   started
   talking about modifying client to not know about ZK, but rather
   keep
   the
   location of current masters and talk to them using RPC calls.
 This
   work
   can
   not go into branch-1, as it involves invasive changes in client
   including
   new RPC. As I understand the branching schema now, those changes
   can
   go
   to
   master branch, we just don't merge them to branch-1, and
 depending
   on
   their
   completeness we can pull them to 1.1 release or so.
  
   You have it right Mikhail.
  
   St.Ack
  
  
  
   --
   Thanks,
   Michael Antonov
  
  
  
   --
   Thanks,
   Michael Antonov
 




-- 
// Jonathan Hsieh (shay)
// HBase Tech Lead, Software Engineer, Cloudera
// j...@cloudera.com // @jmhsieh


[jira] [Created] (HBASE-11474) [Thrift2] support authentication/impersonation

2014-07-07 Thread Jimmy Xiang (JIRA)
Jimmy Xiang created HBASE-11474:
---

 Summary: [Thrift2] support authentication/impersonation
 Key: HBASE-11474
 URL: https://issues.apache.org/jira/browse/HBASE-11474
 Project: HBase
  Issue Type: Improvement
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


This is to do the same as HBASE-11349 for Thrift2.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


Jenkins Build Slaves

2014-07-07 Thread Giridharan Kesavan
All

Yahoo is in the process of retiring all the hadoop jenkins build slaves,
*hadoop[1-9]* and
replace them with a newer set of beefier hosts. These new machines are
configured
with *ubuntu-14.04*.

Over the next couple of days I will be configuring the build jobs to run on
these newly
configured build slaves.  To automate the installation of tools and build
libraries I have
put together ansible scripts and here is the link to the toolchain repo.


*https://github.com/apache/toolchain https://github.com/apache/toolchain*

During the transition, the old build slave will be accessible, and
expected to be shutdown by 07/15.

I will send out an update later this week when this transition is complete.

*Mean while, I would like to request the project owners to remove/cleanup
any stale *
*jenkins job for their respective project and help with any builds issue to
make this *
*transition seamless. *

Thanks

-
Giri

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.


[jira] [Created] (HBASE-11475) Distributed log replay should also replay compaction events

2014-07-07 Thread Enis Soztutar (JIRA)
Enis Soztutar created HBASE-11475:
-

 Summary: Distributed log replay should also replay compaction 
events
 Key: HBASE-11475
 URL: https://issues.apache.org/jira/browse/HBASE-11475
 Project: HBase
  Issue Type: Bug
Reporter: Enis Soztutar
Assignee: Enis Soztutar
 Fix For: 0.99.0, 0.98.4, 2.0.0


Compaction events are written to WAL as of HBASE-2231 got committed. However, 
it seems that distributed log replay skips those entries without sending it to 
the replaying region. In contrast log split replays those. 

It seems that we should fix log replay to also replay the compaction events.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Reopened] (HBASE-11293) Master and Region servers fail to start when hbase.master.ipc.address=0.0.0.0, hbase.regionserver.ipc.address=0.0.0.0 and Kerberos is enabled

2014-07-07 Thread Devaraj Das (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das reopened HBASE-11293:
-


[~miharp], this is not done yet. Trunk needs to have a patch.

 Master and Region servers fail to start when 
 hbase.master.ipc.address=0.0.0.0, hbase.regionserver.ipc.address=0.0.0.0 and 
 Kerberos is enabled
 -

 Key: HBASE-11293
 URL: https://issues.apache.org/jira/browse/HBASE-11293
 Project: HBase
  Issue Type: Bug
Reporter: Michael Harp
Assignee: Devaraj Das
 Attachments: 11293-1.txt


 Setting 
 {code}
 hbase.master.ipc.address=0.0.0.0
 hbase.regionserver.ipc.address=0.0.0.0
 {code}
 causes the _HOST substitution in hbase/_h...@example.com to result in 
 hbase/0:0:0:0:0:0:0:0...@example.com which in turn causes kerberos 
 authentication to fail.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (HBASE-11476) Expand 'Conceptual View' section of Data Model chapter

2014-07-07 Thread Misty Stanley-Jones (JIRA)
Misty Stanley-Jones created HBASE-11476:
---

 Summary: Expand 'Conceptual View' section of Data Model chapter 
 Key: HBASE-11476
 URL: https://issues.apache.org/jira/browse/HBASE-11476
 Project: HBase
  Issue Type: Bug
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones


Could use some updating and expansion to emphasize the differences between 
HBase and an RDBMS. I found 
http://jimbojw.com/wiki/index.php?title=Understanding_Hbase_and_BigTable which 
is just excellent and we should link to it.



--
This message was sent by Atlassian JIRA
(v6.2#6252)