Re: [VOTE] First release candidate for Apache HBase 1.4.12

[Karthik Palanisamy]

Hi Sean,

Please ignore my previous observation regarding MOB failure. I do not see
MOB code in branch-1.x.  HBase support MOB from branch-2.x. Realized HDP
and CDH has backported MOB feature to branch-1.x but I am very curious
hbase-1.2.x doc talks about the MOB feature.
http://hbase.apache.org/1.2/book.html#hbase_mob

Thanks,
Karthik

On Wed, Nov 27, 2019 at 7:15 PM Karthik Palanisamy 
wrote:

> +1 (non-binding)
>
> Setup:
> rhel 7 (3.10.0-123.9.3.el7.x86_64) & jdk1.8.0_211
> hbase-1.4.12RC0 installed via local build
> Fully distributed secure mode: 1 master and 3 regionservers
> - Maven repo artifacts:  ok
> - Signature: ok
> - Checksum: failed
> - Rat check (jdk1.8.0_211): ok
> - Built from source (jdk1.8.0_211): ok
> - Unit tests pass (jdk1.8.0_211):
> - Admin cli
> namespace:  ok
> ddl:  ok
> dml:  ok
> general:  ok
> tools:  ok
> snapshots:  ok
> configuration:  ok
> quotas:  ok
> procedures:  ok
> security:  ok
> visibility labels:  ok
> rsgroup:  ok
> - Webui:  ok
> - Secure thrift:  ok
> - Secure rest:  ok
> - LTT (20 million kvs):  ok
>
> Observation:
> [CLI]  Create MOB table command is broken
> hbase(main):004:0> create 't1', {NAME => 'f1', IS_MOB => true,
> MOB_THRESHOLD => 102400}
> NameError: uninitialized constant IS_MOB
>
> - Karthik
>
> On Wed, Nov 27, 2019 at 3:57 AM Guangxu Cheng 
> wrote:
>
>> +1 (non-binding)
>>   Ran the vote script and the result is as follows. The reason why
>> checksum
>> failed is that the format of
>> hbase-1.4.12-bin.tar.gz.sha512/hbase-1.4.12-src.tar.gz.sha512 are wrong,
>> checksum matching
>>
>>   * Signature: ok
>>   * Checksum : failed
>>   * Rat check (1.8.0_201): ok
>>- mvn clean apache-rat:check
>>   * Built from source (1.8.0_201): ok
>>- mvn clean install -DskipTests
>>   * Unit tests pass (1.8.0_201): ok
>>- mvn package -P runAllTests
>>
>>   * Start in standalone mode: ok
>>   * Basic shell commands (CRUD/flush/compact/split): ok
>>   * LTT 2M rows:  ok
>>   * Basic WEB UI: ok
>>
>> Jan Hentschel  于2019年11月27日周三 上午2:40写道：
>>
>> > +1 (binding)
>> >
>> > Ran the vote script and had the same problem/solution with the checksum
>> > described by Wellington below.
>> >
>> > * Signature: ok
>> > * Checksum : failed
>> > * Rat check (1.8.0_202-ea): ok
>> >  - mvn clean apache-rat:check
>> > * Built from source (1.8.0_202-ea): ok
>> >  - mvn clean install -DskipTests
>> > * Unit tests pass (1.8.0_202-ea): ok
>> >  - mvn package -P runSmallTests
>> >
>> > Best, Jan
>> >
>> > From: Wellington Chevreuil 
>> > Reply-To: "dev@hbase.apache.org" 
>> > Date: Tuesday, November 26, 2019 at 2:32 PM
>> > To: "dev@hbase.apache.org" 
>> > Subject: Re: [VOTE] First release candidate for Apache HBase 1.4.12
>> >
>> > +1
>> > Downloaded source tar, unpacked, then running hbase-vote script all
>> passed,
>> > apart from checksum failure, but I then had checksum manually and it is
>> > matching.
>> > * Signature: ok
>> > * Checksum : failed
>> > * Rat check (1.8.0_222): ok
>> >  - mvn clean apache-rat:check
>> > * Built from source (1.8.0_222): ok
>> >  - mvn clean install -DskipTests
>> > * Unit tests pass (1.8.0_222): ok
>> >  - mvn package -P runSmallTests
>> > Built from source on pseudo distributed mode:
>> > CRUD operations: ok
>> > ltt 10,000,000: ok
>> > Web UI: ok
>> >
>> > Em dom., 24 de nov. de 2019 às 20:54, Sean Busbey > > >
>> > escreveu:
>> >
>> > The first release candidate for HBase 1.4.12 is available for download:
>> >
>> > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.12RC0/
>> >
>> > Maven artifacts are also available in a staging repository at:
>> >
>> > https://repository.apache.org/content/repositories/orgapachehbase-1373/
>> >
>> > Artifacts are signed with my key (0D80DB7C) published in our KEYS
>> > file at http://www.apache.org/dist/hbase/KEYS
>> >
>> > The RC corresponds to the signed tag 1.4.12RC0, which currently points
>> > to commit ref
>> >
>> > 6ae4a77408ad35d6a7a4e5cebfd401fc4b72b5ec
>> >
>> > HBase 1.4.12 is the next maintenance release in the HBase 1.4 line,
>> > continuing on the theme of bringing a stable, reliable database to the
>> > Hadoop and NoSQL communities. This release includes about a dozen
>> > fixes and improvements done by the HBase community in the five
>> > weeks since the prior release.
>> >
>> > The detailed source and binary compatibility report vs 1.4.11 has been
>> > published for your review, at:
>> >
>> > https://s.apache.org/hbase-1.4.12-rc0-compat-report
>> >
>> > The report shows no issues.
>> >
>> > Critical changes include:
>> >
>> > * HBASE-23227 - Upgrade jackson-databind to 2.9.10.1 to avoid recent
>> >  CVEs
>> > * HBASE-23287 - WALs not 

Re: [VOTE] First release candidate for Apache HBase 1.4.12

[Karthik Palanisamy]

+1 (non-binding)

Setup:
rhel 7 (3.10.0-123.9.3.el7.x86_64) & jdk1.8.0_211
hbase-1.4.12RC0 installed via local build
Fully distributed secure mode: 1 master and 3 regionservers
- Maven repo artifacts:  ok
- Signature: ok
- Checksum: failed
- Rat check (jdk1.8.0_211): ok
- Built from source (jdk1.8.0_211): ok
- Unit tests pass (jdk1.8.0_211):
- Admin cli
namespace:  ok
ddl:  ok
dml:  ok
general:  ok
tools:  ok
snapshots:  ok
configuration:  ok
quotas:  ok
procedures:  ok
security:  ok
visibility labels:  ok
rsgroup:  ok
- Webui:  ok
- Secure thrift:  ok
- Secure rest:  ok
- LTT (20 million kvs):  ok

Observation:
[CLI]  Create MOB table command is broken
hbase(main):004:0> create 't1', {NAME => 'f1', IS_MOB => true,
MOB_THRESHOLD => 102400}
NameError: uninitialized constant IS_MOB

- Karthik

On Wed, Nov 27, 2019 at 3:57 AM Guangxu Cheng 
wrote:

> +1 (non-binding)
>   Ran the vote script and the result is as follows. The reason why checksum
> failed is that the format of
> hbase-1.4.12-bin.tar.gz.sha512/hbase-1.4.12-src.tar.gz.sha512 are wrong,
> checksum matching
>
>   * Signature: ok
>   * Checksum : failed
>   * Rat check (1.8.0_201): ok
>- mvn clean apache-rat:check
>   * Built from source (1.8.0_201): ok
>- mvn clean install -DskipTests
>   * Unit tests pass (1.8.0_201): ok
>- mvn package -P runAllTests
>
>   * Start in standalone mode: ok
>   * Basic shell commands (CRUD/flush/compact/split): ok
>   * LTT 2M rows:  ok
>   * Basic WEB UI: ok
>
> Jan Hentschel  于2019年11月27日周三 上午2:40写道：
>
> > +1 (binding)
> >
> > Ran the vote script and had the same problem/solution with the checksum
> > described by Wellington below.
> >
> > * Signature: ok
> > * Checksum : failed
> > * Rat check (1.8.0_202-ea): ok
> >  - mvn clean apache-rat:check
> > * Built from source (1.8.0_202-ea): ok
> >  - mvn clean install -DskipTests
> > * Unit tests pass (1.8.0_202-ea): ok
> >  - mvn package -P runSmallTests
> >
> > Best, Jan
> >
> > From: Wellington Chevreuil 
> > Reply-To: "dev@hbase.apache.org" 
> > Date: Tuesday, November 26, 2019 at 2:32 PM
> > To: "dev@hbase.apache.org" 
> > Subject: Re: [VOTE] First release candidate for Apache HBase 1.4.12
> >
> > +1
> > Downloaded source tar, unpacked, then running hbase-vote script all
> passed,
> > apart from checksum failure, but I then had checksum manually and it is
> > matching.
> > * Signature: ok
> > * Checksum : failed
> > * Rat check (1.8.0_222): ok
> >  - mvn clean apache-rat:check
> > * Built from source (1.8.0_222): ok
> >  - mvn clean install -DskipTests
> > * Unit tests pass (1.8.0_222): ok
> >  - mvn package -P runSmallTests
> > Built from source on pseudo distributed mode:
> > CRUD operations: ok
> > ltt 10,000,000: ok
> > Web UI: ok
> >
> > Em dom., 24 de nov. de 2019 às 20:54, Sean Busbey  > >
> > escreveu:
> >
> > The first release candidate for HBase 1.4.12 is available for download:
> >
> > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.12RC0/
> >
> > Maven artifacts are also available in a staging repository at:
> >
> > https://repository.apache.org/content/repositories/orgapachehbase-1373/
> >
> > Artifacts are signed with my key (0D80DB7C) published in our KEYS
> > file at http://www.apache.org/dist/hbase/KEYS
> >
> > The RC corresponds to the signed tag 1.4.12RC0, which currently points
> > to commit ref
> >
> > 6ae4a77408ad35d6a7a4e5cebfd401fc4b72b5ec
> >
> > HBase 1.4.12 is the next maintenance release in the HBase 1.4 line,
> > continuing on the theme of bringing a stable, reliable database to the
> > Hadoop and NoSQL communities. This release includes about a dozen
> > fixes and improvements done by the HBase community in the five
> > weeks since the prior release.
> >
> > The detailed source and binary compatibility report vs 1.4.11 has been
> > published for your review, at:
> >
> > https://s.apache.org/hbase-1.4.12-rc0-compat-report
> >
> > The report shows no issues.
> >
> > Critical changes include:
> >
> > * HBASE-23227 - Upgrade jackson-databind to 2.9.10.1 to avoid recent
> >  CVEs
> > * HBASE-23287 - WALs not aged off of HDFS because LogCleaner is not
> >  added to choreService
> >
> > The full set of fixes in this release can be found here:
> >
> > https://s.apache.org/hbase-1.4.12-jira-release-notes
> >
> > and in the CHANGES.txt file included in the distribution.
> >
> > Please try out this candidate and vote +1/-1 on whether we should
> > release these artifacts as Apache HBase 1.4.12.
> >
> > The VOTE will remain open for at least 72 hours.
> >
> > Thanks!
> >
> > --busbey
> >
> > as of this email the posted artifacts have the following SHA512:
> >
> > 

[jira] [Resolved] (HBASE-23293) [REPLICATION] make ship edits timeout configurable

[Guangxu Cheng (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guangxu Cheng resolved HBASE-23293.
---
Fix Version/s: 2.2.3
   2.3.0
   3.0.0
   Resolution: Fixed

> [REPLICATION] make ship edits timeout configurable
> --
>
> Key: HBASE-23293
> URL: https://issues.apache.org/jira/browse/HBASE-23293
> Project: HBase
>  Issue Type: Improvement
>  Components: Replication
>Reporter: chenxu
>Assignee: chenxu
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3
>
>
> ReplicationSourceShipper#shipEdits may take a while if bulkload replication 
> enabled, since we should copy HFile from the source cluster, so i think there 
> is a need to make the timeout value configurable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-23347) Pluggable RPC authentication

[Josh Elser (Jira)]

Josh Elser created HBASE-23347:
--

 Summary: Pluggable RPC authentication
 Key: HBASE-23347
 URL: https://issues.apache.org/jira/browse/HBASE-23347
 Project: HBase
  Issue Type: Improvement
  Components: rpc, security
Reporter: Josh Elser
Assignee: Josh Elser
 Fix For: 3.0.0


Today in HBase, we rely on SASL to implement Kerberos and delegation token 
authentication. The RPC client and server logic is very tightly coupled to our 
three authentication mechanism (the previously two mentioned plus simple 
auth'n) for no good reason (other than "that's how it was built", best as I can 
tell).

SASL's function is to decouple the "application" from how a request is being 
authenticated, which means that, to support a variety of other authentication 
approaches, we just need to be a little more flexible in letting developers 
create their own authentication mechanism for HBase.

This is less for the "average joe" user to write their own authentication 
plugin (eek), but more to allow us HBase developers to start iterating, see 
what is possible.

I'll attach a full write-up on what I have today as to how I think we can add 
these abstractions, as well as an initial implementation of this idea, with a 
unit test that shows an end-to-end authentication solution against HBase.

cc/ [~wchevreuil] as he's been working with me behind the scenes, giving lots 
of great feedback and support.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-23313) [hbck2] setRegionState should update Master in-memory state too

[Wellington Chevreuil (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wellington Chevreuil resolved HBASE-23313.
--
Resolution: Fixed

Pushed to master an branch-2 branches. There were few RPC changes, only new 
methods and attributes added.

> [hbck2] setRegionState should update Master in-memory state too
> ---
>
> Key: HBASE-23313
> URL: https://issues.apache.org/jira/browse/HBASE-23313
> Project: HBase
>  Issue Type: Bug
>  Components: hbck2
>Affects Versions: 2.1.7, 2.2.2
>Reporter: Michael Stack
>Assignee: Wellington Chevreuil
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> setRegionState changes the hbase:meta table info:state column. It does not 
> alter the Master's in-memory state. This means you have to kill Master and 
> have another assume Active Master role of a state-change to be noticed. 
> Better if the setRegionState just went via Master and updated Master and 
> hbase:meta.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] First release candidate for Apache HBase 1.4.12

[Guangxu Cheng]

+1 (non-binding)
  Ran the vote script and the result is as follows. The reason why checksum
failed is that the format of
hbase-1.4.12-bin.tar.gz.sha512/hbase-1.4.12-src.tar.gz.sha512 are wrong,
checksum matching

  * Signature: ok
  * Checksum : failed
  * Rat check (1.8.0_201): ok
   - mvn clean apache-rat:check
  * Built from source (1.8.0_201): ok
   - mvn clean install -DskipTests
  * Unit tests pass (1.8.0_201): ok
   - mvn package -P runAllTests

  * Start in standalone mode: ok
  * Basic shell commands (CRUD/flush/compact/split): ok
  * LTT 2M rows:  ok
  * Basic WEB UI: ok

Jan Hentschel  于2019年11月27日周三 上午2:40写道：

> +1 (binding)
>
> Ran the vote script and had the same problem/solution with the checksum
> described by Wellington below.
>
> * Signature: ok
> * Checksum : failed
> * Rat check (1.8.0_202-ea): ok
>  - mvn clean apache-rat:check
> * Built from source (1.8.0_202-ea): ok
>  - mvn clean install -DskipTests
> * Unit tests pass (1.8.0_202-ea): ok
>  - mvn package -P runSmallTests
>
> Best, Jan
>
> From: Wellington Chevreuil 
> Reply-To: "dev@hbase.apache.org" 
> Date: Tuesday, November 26, 2019 at 2:32 PM
> To: "dev@hbase.apache.org" 
> Subject: Re: [VOTE] First release candidate for Apache HBase 1.4.12
>
> +1
> Downloaded source tar, unpacked, then running hbase-vote script all passed,
> apart from checksum failure, but I then had checksum manually and it is
> matching.
> * Signature: ok
> * Checksum : failed
> * Rat check (1.8.0_222): ok
>  - mvn clean apache-rat:check
> * Built from source (1.8.0_222): ok
>  - mvn clean install -DskipTests
> * Unit tests pass (1.8.0_222): ok
>  - mvn package -P runSmallTests
> Built from source on pseudo distributed mode:
> CRUD operations: ok
> ltt 10,000,000: ok
> Web UI: ok
>
> Em dom., 24 de nov. de 2019 às 20:54, Sean Busbey  >
> escreveu:
>
> The first release candidate for HBase 1.4.12 is available for download:
>
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.12RC0/
>
> Maven artifacts are also available in a staging repository at:
>
> https://repository.apache.org/content/repositories/orgapachehbase-1373/
>
> Artifacts are signed with my key (0D80DB7C) published in our KEYS
> file at http://www.apache.org/dist/hbase/KEYS
>
> The RC corresponds to the signed tag 1.4.12RC0, which currently points
> to commit ref
>
> 6ae4a77408ad35d6a7a4e5cebfd401fc4b72b5ec
>
> HBase 1.4.12 is the next maintenance release in the HBase 1.4 line,
> continuing on the theme of bringing a stable, reliable database to the
> Hadoop and NoSQL communities. This release includes about a dozen
> fixes and improvements done by the HBase community in the five
> weeks since the prior release.
>
> The detailed source and binary compatibility report vs 1.4.11 has been
> published for your review, at:
>
> https://s.apache.org/hbase-1.4.12-rc0-compat-report
>
> The report shows no issues.
>
> Critical changes include:
>
> * HBASE-23227 - Upgrade jackson-databind to 2.9.10.1 to avoid recent
>  CVEs
> * HBASE-23287 - WALs not aged off of HDFS because LogCleaner is not
>  added to choreService
>
> The full set of fixes in this release can be found here:
>
> https://s.apache.org/hbase-1.4.12-jira-release-notes
>
> and in the CHANGES.txt file included in the distribution.
>
> Please try out this candidate and vote +1/-1 on whether we should
> release these artifacts as Apache HBase 1.4.12.
>
> The VOTE will remain open for at least 72 hours.
>
> Thanks!
>
> --busbey
>
> as of this email the posted artifacts have the following SHA512:
>
> hbase-1.4.12-src.tar.gz:
> 72A0E4DF BB44C4F3 A07E3FB5 5D6FD3BC DD40C9CC
> BCD311C0 CB61ACB4 2756C735 D465A340 FB05176D
> DDCB29C5 A5D695F6 BC8F8E7E 3AAF6499 8A149DA8
> EC6C4E7A
>
> hbase-1.4.12-bin.tar.gz:
> 58E2873F E4725E0C 6E77437E B09864A5 BD9E50DD
> 9BBE84AD 7461FBCA 8AE51A0C D91DA20B 90E8AD82
> DCF50EB2 C82AC3C8 26CD1C58 8ED69D2A 1E6AD0B4
> 24CD94D5
>
>
>