[jira] [Created] (HBASE-27414) Search order for locations in HFileLink

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-27414:


 Summary: Search order for locations in  HFileLink
 Key: HBASE-27414
 URL: https://issues.apache.org/jira/browse/HBASE-27414
 Project: HBase
  Issue Type: Improvement
  Components: Performance
Reporter: Huaxiang Sun


Found that search order for locations is following the order of these locations 
added to HFileLink object. 

 

setLocations(originPath, tempPath, mobPath, archivePath);

archivePath is the last one to be searched. For most cases, hfile exists in 
archivePath, so we can move archivePath to the first parameter to avoid 
unnecessary NN query.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27366) split or merge removed region under snapshot

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-27366:


 Summary: split or merge removed region under snapshot
 Key: HBASE-27366
 URL: https://issues.apache.org/jira/browse/HBASE-27366
 Project: HBase
  Issue Type: Bug
  Components: snapshots
Affects Versions: 2.4.10
Reporter: Huaxiang Sun


We run into snapshot failures for one table with large number of regions. The 
event sequence is like the following:

 
 # Snapshot process lists all regions for one table.
 # Normalize kicks in to split some regions for the table under snapshot.
 # split finishes and major compaction finishes. The parent region is moved to 
archive.
 # When the Snapshot processes the parent region, it does not exist and 
snapshot fails.

Since snapshot process acquires the table lock, but there is no table lock 
acquired in split or merge process, they crash into each other.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[ANNOUNCE] Apache HBase 2.4.14 is now available for download

[Huaxiang Sun]

The HBase team is happy to announce the immediate availability of HBase

2.4.14.


Apache HBase™ is an open-source, distributed, versioned, non-relational

database.


Apache HBase gives you low latency random access to billions of rows with

millions of columns atop non-specialized hardware. To learn more about

HBase, see https://hbase.apache.org/.


HBase 2.4.14 is the fourteenth patch release in the HBase 2.4.x line, which
aims to improve the stability and reliability of HBase. This release
includes 49 bug fixes and improvements since 2.4.13.


The full list of issues and release notes can be found here:


CHANGELOG: https://downloads.apache.org/hbase/2.4.14/CHANGES.md

RELEASENOTES: https://downloads.apache.org/hbase/2.4.14/RELEASENOTES.md



or via our issue tracker:

https://issues.apache.org/jira/projects/HBASE/2.4.14-jiras



To download please follow the links and instructions on our website:


https://hbase.apache.org/downloads.html


Questions, comments, and problems are always welcome at:


dev@hbase.apache.org

u...@hbase.apache.org


Thanks to all who contributed and made this release possible.


Cheers,

The HBase Dev Team

Re: [VOTE] HBase 2.4.14 release candidate (RC1) is available

[Huaxiang Sun]

Done, thanks for checking.


On Tue, Aug 30, 2022 at 9:06 AM Huaxiang Sun  wrote:

> Thanks Duo, let me confirm before sending out the announcement.
>
> Huaxiang
>
> On Tue, Aug 30, 2022 at 8:23 AM 张铎(Duo Zhang) 
> wrote:
>
>> Seems you forget to release the artifacts on
>> https://repository.apache.org/?
>>
>> Huaxiang Sun  于2022年8月30日周二 13:07写道：
>> >
>> > Thanks Andrew.
>> >
>> > Update:
>> >
>> > With six (6) binding +1 votes, and two (2) nonbinding +1 vote, and no 0
>> or -1 votes, this vote passes.
>> >
>> > I have done most work for this release, once the download page is
>> updated, announcement email will be sent out.
>> >
>> > Huaxiang
>> >
>> > Sent from my iPhone
>> >
>> > > On Aug 29, 2022, at 5:38 PM, Andrew Purtell 
>> wrote:
>> > >
>> > > +1 (binding)
>> > >
>> > >* Signature: ok
>> > >* Checksum : ok
>> > >* Rat check (1.8.0_332): ok
>> > > - mvn clean apache-rat:check
>> > >* Built from source (1.8.0_332): ok
>> > > - mvn clean install  -DskipTests
>> > >* Unit tests pass (1.8.0_332): failed
>> > > - mvn package -P runAllTests
>> -Dsurefire.rerunFailingTestsCount=3
>> > >
>> > > TestSplitRegionWhileRSCrash times out.
>> > >
>> > >
>> > >> On Wed, Aug 24, 2022 at 8:59 AM Huaxiang Sun 
>> wrote:
>> > >>
>> > >> Please vote on this Apache hbase release candidate,
>> > >>
>> > >> hbase-2.4.14RC1
>> > >>
>> > >>
>> > >> The VOTE will remain open for at least 72 hours.
>> > >>
>> > >>
>> > >> [ ] +1 Release this package as Apache hbase 2.4.14
>> > >>
>> > >> [ ] -1 Do not release this package because ...
>> > >>
>> > >>
>> > >> The tag to be voted on is 2.4.14RC1:
>> > >>
>> > >>
>> > >>  https://github.com/apache/hbase/tree/2.4.14RC1
>> > >>
>> > >>
>> > >> This tag currently points to git reference
>> > >>
>> > >>
>> > >>  2e7d75a89271a7479b2f668c4db7a241be3f
>> > >>
>> > >>
>> > >> The release files, including signatures, digests, as well as
>> CHANGES.md
>> > >>
>> > >> and RELEASENOTES.md included in this RC can be found at:
>> > >>
>> > >>
>> > >>  https://dist.apache.org/repos/dist/dev/hbase/2.4.14RC1/
>> > >>
>> > >>
>> > >> Maven artifacts are available in a staging repository at:
>> > >>
>> > >>
>> > >>
>> https://repository.apache.org/content/repositories/orgapachehbase-1495/
>> > >>
>> > >>
>> > >> Artifacts were signed with the 0x117C835E key which can be found in:
>> > >>
>> > >>
>> > >>  https://downloads.apache.org/hbase/KEYS
>> > >>
>> > >>
>> > >> To learn more about Apache hbase, please see
>> > >>
>> > >>
>> > >>  http://hbase.apache.org/
>> > >>
>> > >>
>> > >> Thanks,
>> > >>
>> > >> Your HBase Release Manager
>> > >>
>> > >
>> > >
>> > > --
>> > > Best regards,
>> > > Andrew
>> > >
>> > > Unrest, ignorance distilled, nihilistic imbeciles -
>> > >It's what we’ve earned
>> > > Welcome, apocalypse, what’s taken you so long?
>> > > Bring us the fitting end that we’ve been counting on
>> > >   - A23, Welcome, Apocalypse
>>
>

Re: [VOTE] HBase 2.4.14 release candidate (RC1) is available

[Huaxiang Sun]

Thanks Duo, let me confirm before sending out the announcement.

Huaxiang

On Tue, Aug 30, 2022 at 8:23 AM 张铎(Duo Zhang)  wrote:

> Seems you forget to release the artifacts on
> https://repository.apache.org/?
>
> Huaxiang Sun  于2022年8月30日周二 13:07写道：
> >
> > Thanks Andrew.
> >
> > Update:
> >
> > With six (6) binding +1 votes, and two (2) nonbinding +1 vote, and no 0
> or -1 votes, this vote passes.
> >
> > I have done most work for this release, once the download page is
> updated, announcement email will be sent out.
> >
> > Huaxiang
> >
> > Sent from my iPhone
> >
> > > On Aug 29, 2022, at 5:38 PM, Andrew Purtell 
> wrote:
> > >
> > > +1 (binding)
> > >
> > >* Signature: ok
> > >* Checksum : ok
> > >* Rat check (1.8.0_332): ok
> > > - mvn clean apache-rat:check
> > >* Built from source (1.8.0_332): ok
> > > - mvn clean install  -DskipTests
> > >* Unit tests pass (1.8.0_332): failed
> > > - mvn package -P runAllTests
> -Dsurefire.rerunFailingTestsCount=3
> > >
> > > TestSplitRegionWhileRSCrash times out.
> > >
> > >
> > >> On Wed, Aug 24, 2022 at 8:59 AM Huaxiang Sun 
> wrote:
> > >>
> > >> Please vote on this Apache hbase release candidate,
> > >>
> > >> hbase-2.4.14RC1
> > >>
> > >>
> > >> The VOTE will remain open for at least 72 hours.
> > >>
> > >>
> > >> [ ] +1 Release this package as Apache hbase 2.4.14
> > >>
> > >> [ ] -1 Do not release this package because ...
> > >>
> > >>
> > >> The tag to be voted on is 2.4.14RC1:
> > >>
> > >>
> > >>  https://github.com/apache/hbase/tree/2.4.14RC1
> > >>
> > >>
> > >> This tag currently points to git reference
> > >>
> > >>
> > >>  2e7d75a89271a7479b2f668c4db7a241be3f
> > >>
> > >>
> > >> The release files, including signatures, digests, as well as
> CHANGES.md
> > >>
> > >> and RELEASENOTES.md included in this RC can be found at:
> > >>
> > >>
> > >>  https://dist.apache.org/repos/dist/dev/hbase/2.4.14RC1/
> > >>
> > >>
> > >> Maven artifacts are available in a staging repository at:
> > >>
> > >>
> > >>
> https://repository.apache.org/content/repositories/orgapachehbase-1495/
> > >>
> > >>
> > >> Artifacts were signed with the 0x117C835E key which can be found in:
> > >>
> > >>
> > >>  https://downloads.apache.org/hbase/KEYS
> > >>
> > >>
> > >> To learn more about Apache hbase, please see
> > >>
> > >>
> > >>  http://hbase.apache.org/
> > >>
> > >>
> > >> Thanks,
> > >>
> > >> Your HBase Release Manager
> > >>
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew
> > >
> > > Unrest, ignorance distilled, nihilistic imbeciles -
> > >It's what we’ve earned
> > > Welcome, apocalypse, what’s taken you so long?
> > > Bring us the fitting end that we’ve been counting on
> > >   - A23, Welcome, Apocalypse
>

Re: [VOTE] HBase 2.4.14 release candidate (RC1) is available

[Huaxiang Sun]

Thanks Andrew.

Update:

With six (6) binding +1 votes, and two (2) nonbinding +1 vote, and no 0 or -1 
votes, this vote passes. 

I have done most work for this release, once the download page is updated, 
announcement email will be sent out.

Huaxiang

Sent from my iPhone

> On Aug 29, 2022, at 5:38 PM, Andrew Purtell  wrote:
> 
> +1 (binding)
> 
>* Signature: ok
>* Checksum : ok
>* Rat check (1.8.0_332): ok
> - mvn clean apache-rat:check
>* Built from source (1.8.0_332): ok
> - mvn clean install  -DskipTests
>* Unit tests pass (1.8.0_332): failed
> - mvn package -P runAllTests  -Dsurefire.rerunFailingTestsCount=3
> 
> TestSplitRegionWhileRSCrash times out.
> 
> 
>> On Wed, Aug 24, 2022 at 8:59 AM Huaxiang Sun  wrote:
>> 
>> Please vote on this Apache hbase release candidate,
>> 
>> hbase-2.4.14RC1
>> 
>> 
>> The VOTE will remain open for at least 72 hours.
>> 
>> 
>> [ ] +1 Release this package as Apache hbase 2.4.14
>> 
>> [ ] -1 Do not release this package because ...
>> 
>> 
>> The tag to be voted on is 2.4.14RC1:
>> 
>> 
>>  https://github.com/apache/hbase/tree/2.4.14RC1
>> 
>> 
>> This tag currently points to git reference
>> 
>> 
>>  2e7d75a89271a7479b2f668c4db7a241be3f
>> 
>> 
>> The release files, including signatures, digests, as well as CHANGES.md
>> 
>> and RELEASENOTES.md included in this RC can be found at:
>> 
>> 
>>  https://dist.apache.org/repos/dist/dev/hbase/2.4.14RC1/
>> 
>> 
>> Maven artifacts are available in a staging repository at:
>> 
>> 
>>  https://repository.apache.org/content/repositories/orgapachehbase-1495/
>> 
>> 
>> Artifacts were signed with the 0x117C835E key which can be found in:
>> 
>> 
>>  https://downloads.apache.org/hbase/KEYS
>> 
>> 
>> To learn more about Apache hbase, please see
>> 
>> 
>>  http://hbase.apache.org/
>> 
>> 
>> Thanks,
>> 
>> Your HBase Release Manager
>> 
> 
> 
> -- 
> Best regards,
> Andrew
> 
> Unrest, ignorance distilled, nihilistic imbeciles -
>It's what we’ve earned
> Welcome, apocalypse, what’s taken you so long?
> Bring us the fitting end that we’ve been counting on
>   - A23, Welcome, Apocalypse

[jira] [Resolved] (HBASE-27345) Add 2.4.14 to the downloads page

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-27345.
--
Fix Version/s: 3.0.0-alpha-4
 Assignee: Huaxiang Sun
   Resolution: Fixed

> Add 2.4.14 to the downloads page
> 
>
> Key: HBASE-27345
> URL: https://issues.apache.org/jira/browse/HBASE-27345
> Project: HBase
>  Issue Type: Task
>  Components: documentation
>Affects Versions: 2.4.14
>    Reporter: Huaxiang Sun
>    Assignee: Huaxiang Sun
>Priority: Minor
> Fix For: 3.0.0-alpha-4
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27345) Add 2.4.14 to the downloads page

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-27345:


 Summary: Add 2.4.14 to the downloads page
 Key: HBASE-27345
 URL: https://issues.apache.org/jira/browse/HBASE-27345
 Project: HBase
  Issue Type: Task
  Components: documentation
Affects Versions: 2.4.14
Reporter: Huaxiang Sun






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] HBase 2.4.14 release candidate (RC1) is available

[Huaxiang Sun]

With five (5) binding +1 votes, and two (2) nonbinding +1 vote, and no 0 or
-1 votes, this vote passes.


Thanks all for the vote.


Huaxiang

On Sat, Aug 27, 2022 at 9:55 AM Jan Hentschel
 wrote:

> +1 (binding)
>
> * Signature: ok
> * Checksum : ok
> * Rat check (1.8.0_202-ea): ok
>  - mvn clean apache-rat:check
> * Built from source (1.8.0_202-ea): ok
>  - mvn clean install  -DskipTests
> * Unit tests pass (1.8.0_202-ea): ok
>  - mvn package -P runSmallTests
> -Dsurefire.rerunFailingTestsCount=3
>
> From: Huaxiang Sun 
> Date: Wednesday, August 24, 2022 at 5:59 PM
> To: hbase-dev 
> Subject: [VOTE] HBase 2.4.14 release candidate (RC1) is available
> [CAUTION] The sender of this email is from outside our organization.
> Please DO NOT CLICK any links, download attachments or respond unless you
> recognize the sender and know the content is safe. Please contact IT
> immediately in case you find it suspicious.
>
> Please vote on this Apache hbase release candidate,
>
> hbase-2.4.14RC1
>
>
> The VOTE will remain open for at least 72 hours.
>
>
> [ ] +1 Release this package as Apache hbase 2.4.14
>
> [ ] -1 Do not release this package because ...
>
>
> The tag to be voted on is 2.4.14RC1:
>
>
>
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fhbase%2Ftree%2F2.4.14RC1data=05%7C01%7Cjan.hentschel%40ultratendency.com%7Cc0f1c6604c2441bf698208da85e9a4b3%7C8b2d44831cfd4f70b345dec07bd4f22f%7C0%7C0%7C637969535780563549%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=WXQ9FpS7wTYbTM%2B4qzjw7IFLTAjH51gMX%2BTDerymYFM%3Dreserved=0
>
>
> This tag currently points to git reference
>
>
>   2e7d75a89271a7479b2f668c4db7a241be3f
>
>
> The release files, including signatures, digests, as well as CHANGES.md
>
> and RELEASENOTES.md included in this RC can be found at:
>
>
>
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fhbase%2F2.4.14RC1%2Fdata=05%7C01%7Cjan.hentschel%40ultratendency.com%7Cc0f1c6604c2441bf698208da85e9a4b3%7C8b2d44831cfd4f70b345dec07bd4f22f%7C0%7C0%7C637969535780563549%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=2LagAeb%2FCv0U2yktrTPFM5yNQOdaTJBHAoreUkvY7Uw%3Dreserved=0
>
>
> Maven artifacts are available in a staging repository at:
>
>
>
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachehbase-1495%2Fdata=05%7C01%7Cjan.hentschel%40ultratendency.com%7Cc0f1c6604c2441bf698208da85e9a4b3%7C8b2d44831cfd4f70b345dec07bd4f22f%7C0%7C0%7C637969535780563549%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=gVKb99TUkHulfnYNIMgnIdllOV9t41%2Ffb5gyaQhQ%2F2U%3Dreserved=0
>
>
> Artifacts were signed with the 0x117C835E key which can be found in:
>
>
>
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fhbase%2FKEYSdata=05%7C01%7Cjan.hentschel%40ultratendency.com%7Cc0f1c6604c2441bf698208da85e9a4b3%7C8b2d44831cfd4f70b345dec07bd4f22f%7C0%7C0%7C637969535780563549%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=HGrGgxPH6vlnHgVPJKIF7kP1eemT2f6aJ1UVaWp84%2Fk%3Dreserved=0
>
>
> To learn more about Apache hbase, please see
>
>
>
> https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhbase.apache.org%2Fdata=05%7C01%7Cjan.hentschel%40ultratendency.com%7Cc0f1c6604c2441bf698208da85e9a4b3%7C8b2d44831cfd4f70b345dec07bd4f22f%7C0%7C0%7C637969535780563549%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=R1nky03mcwjy%2FgYVOSl716Kch3PXYtJmhYARLIuAJz4%3Dreserved=0
>
>
> Thanks,
>
> Your HBase Release Manager
>

Re: [VOTE] Release candidate for HBase 2.5.0 (RC1) is available

[Huaxiang Sun]

+1

~/work/hbase-hs/hbase-1

* Signature: ok

* Checksum : failed

* Rat check (1.8.0_242): ok

 - mvn clean apache-rat:check

* Built from source (1.8.0_242): ok

 - mvn clean install  -DskipTests

* Unit tests pass (1.8.0_242): ok

 - mvn package -P runSmallTests  -Dsurefire.rerunFailingTestsCount=3

$ ./dev-support/hbase-vote.sh -s
https://dist.apache.org/repos/dist/dev/hbase/2.5.0RC1/ -P runSmallTests -o
~/hbase-2.5.0-vote


Also run


./bin/hbase ltt -num_keys 200 -write 10:2000:16 -num_tables 3


twice on a local cluster to trigger region split and compaction. It
completed successfully.

Also run into Region Visualizer's undefined region server issue.


Huaxiang



On Fri, Aug 26, 2022 at 12:52 PM Viraj Jasani  wrote:

> +0
>
> * Signature: ok
> * Checksum : ok
> * Rat check (1.8.0_341): ok
>  - mvn clean apache-rat:check
> * Built from source (1.8.0_341): ok
>  - mvn clean install  -DskipTests
> * Unit tests pass (1.8.0_341): failed (TestUsersOperationsWithSecureHadoop
> has known failure locally)
>  - mvn package -P runSmallTests  -Dsurefire.rerunFailingTestsCount=3
> * A bit of high load testing locally looks good
>
> The compat report has few high sev problems reported but not all need to be
> taken care of given that majority are affecting deprecated/private APIs.
> However, there seems one high sev problem that we might want to consider
> fixing:
>
>
> org/apache/hadoop/hbase/io/encoding/DataBlockEncoding.createEncoder:(Ljava/lang/String;)Lorg/apache/hadoop/hbase/io/encoding/DataBlockEncoder
> Access level has been changed from protected to package-private.
>
> This was done as part of the recent error-prone findings. Either we can
> bring back protected access or, we already have public getter
> *getEncoder()*,
> hence I think perhaps we can make *createEncoder(String
> fullyQualifiedClassName)* as IA.Private. Either way is fine.
>
> Functional testing seems good, thanks Nick.
>
>
> On Wed, Aug 24, 2022 at 2:53 AM Nick Dimiduk  wrote:
>
> > Please vote on this Apache hbase release candidate,
> > hbase-2.5.0RC1
> >
> > The VOTE will remain open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache hbase 2.5.0
> > [ ] -1 Do not release this package because ...
> >
> > The tag to be voted on is 2.5.0RC1:
> >
> >   https://github.com/apache/hbase/tree/2.5.0RC1
> >
> > This tag currently points to git reference
> >
> >   2ecd8bd6d615ca49bfb329b3c0c126c80846d4ab
> >
> > The release files, including signatures, digests, as well as CHANGES.md
> > and RELEASENOTES.md included in this RC can be found at:
> >
> >   https://dist.apache.org/repos/dist/dev/hbase/2.5.0RC1/
> >
> > Maven artifacts are available in a staging repository at:
> >
> >
> https://repository.apache.org/content/repositories/orgapachehbase-1494/
> >
> > Artifacts were signed with the 0x18567F39 key which can be found in:
> >
> >   https://downloads.apache.org/hbase/KEYS
> >
> > To learn more about Apache hbase, please see
> >
> >   http://hbase.apache.org/
> >
> > Thanks,
> > Your HBase Release Manager
> >
>

Re: [VOTE] HBase 2.4.14 release candidate (RC1) is available

[Huaxiang Sun]

+1 (binding)

~/work/hbase-hs/hbase-1

* Signature: ok

* Checksum : ok

* Rat check (1.8.0_242): ok

 - mvn clean apache-rat:check

* Built from source (1.8.0_242): ok

 - mvn clean install  -DskipTests

* Unit tests pass (1.8.0_242): ok

 - mvn package -P runSmallTests  -Dsurefire.rerunFailingTestsCount=3



Went through CHANGES.md to make sure all 2.4.14 jiras are included.

On Wed, Aug 24, 2022 at 8:59 AM Huaxiang Sun  wrote:

> Please vote on this Apache hbase release candidate,
>
> hbase-2.4.14RC1
>
>
> The VOTE will remain open for at least 72 hours.
>
>
> [ ] +1 Release this package as Apache hbase 2.4.14
>
> [ ] -1 Do not release this package because ...
>
>
> The tag to be voted on is 2.4.14RC1:
>
>
>   https://github.com/apache/hbase/tree/2.4.14RC1
>
>
> This tag currently points to git reference
>
>
>   2e7d75a89271a7479b2f668c4db7a241be3f
>
>
> The release files, including signatures, digests, as well as CHANGES.md
>
> and RELEASENOTES.md included in this RC can be found at:
>
>
>   https://dist.apache.org/repos/dist/dev/hbase/2.4.14RC1/
>
>
> Maven artifacts are available in a staging repository at:
>
>
>   https://repository.apache.org/content/repositories/orgapachehbase-1495/
>
>
> Artifacts were signed with the 0x117C835E key which can be found in:
>
>
>   https://downloads.apache.org/hbase/KEYS
>
>
> To learn more about Apache hbase, please see
>
>
>   http://hbase.apache.org/
>
>
> Thanks,
>
> Your HBase Release Manager
>

[VOTE] HBase 2.4.14 release candidate (RC1) is available

[Huaxiang Sun]

Please vote on this Apache hbase release candidate,

hbase-2.4.14RC1


The VOTE will remain open for at least 72 hours.


[ ] +1 Release this package as Apache hbase 2.4.14

[ ] -1 Do not release this package because ...


The tag to be voted on is 2.4.14RC1:


  https://github.com/apache/hbase/tree/2.4.14RC1


This tag currently points to git reference


  2e7d75a89271a7479b2f668c4db7a241be3f


The release files, including signatures, digests, as well as CHANGES.md

and RELEASENOTES.md included in this RC can be found at:


  https://dist.apache.org/repos/dist/dev/hbase/2.4.14RC1/


Maven artifacts are available in a staging repository at:


  https://repository.apache.org/content/repositories/orgapachehbase-1495/


Artifacts were signed with the 0x117C835E key which can be found in:


  https://downloads.apache.org/hbase/KEYS


To learn more about Apache hbase, please see


  http://hbase.apache.org/


Thanks,

Your HBase Release Manager

Re: [VOTE] HBase 2.4.14 release candidate (RC0) is available

[Huaxiang Sun]

s/HBASE-27215/HBASE-27125 in the previous email I sent.

On Tue, Aug 23, 2022 at 9:26 AM Huaxiang Sun  wrote:

> Ok, I went through the jira list for 2.4.14 again. For some reason, I
> marked "fixed version" for HBASE-27301 by accident, which was supposed to
> be
>
> HBASE-27125. For HBASE-27215, it was released in 2.4.13, however, its
> "fixed version" is marked as 2.4.14 by accident.
>
> So HBASE-27215 was not included in 2.4.13's release notes.
>
>
> I corrected both jira's fixed versions and respining a new RC.
>
>
>
>
> On Tue, Aug 23, 2022 at 8:44 AM Huaxiang Sun 
> wrote:
>
>> Sorry to miss your message, checking it now, I may marked the wrong jira,
>> let me check the jira list and respin RC.
>>
>> Thanks
>> Huaxiang
>>
>> On Mon, Aug 22, 2022 at 6:41 PM 张铎(Duo Zhang) 
>> wrote:
>>
>>> Sorry but -1.
>>>
>>> I replied on HBASE-27301 that the issue should be included in 2.4.14
>>> but got no response.
>>>
>>> And I checked the CHANGES.md, there is no HBASE-27301 in it, neither
>>> in 2.4.13 nor 2.4.14.
>>>
>>> Please fix this issue and publish an RC again.
>>>
>>> Thanks.
>>>
>>> Huaxiang Sun  于2022年8月23日周二 07:15写道：
>>> >
>>> > Please vote on this Apache hbase release candidate,
>>> >
>>> > hbase-2.4.14RC0
>>> >
>>> >
>>> > The VOTE will remain open for at least 72 hours.
>>> >
>>> >
>>> > [ ] +1 Release this package as Apache hbase 2.4.14
>>> >
>>> > [ ] -1 Do not release this package because ...
>>> >
>>> >
>>> > The tag to be voted on is 2.4.14RC0:
>>> >
>>> >
>>> >   https://github.com/apache/hbase/tree/2.4.14RC0
>>> >
>>> >
>>> > This tag currently points to git reference
>>> >
>>> >
>>> >   f507fb71c6724ed44eb4ac25bbdd3e35b2441a1a
>>> >
>>> >
>>> > The release files, including signatures, digests, as well as CHANGES.md
>>> >
>>> > and RELEASENOTES.md included in this RC can be found at:
>>> >
>>> >
>>> >   https://dist.apache.org/repos/dist/dev/hbase/2.4.14RC0/
>>> >
>>> >
>>> > Maven artifacts are available in a staging repository at:
>>> >
>>> >
>>> >
>>> https://repository.apache.org/content/repositories/orgapachehbase-1493/
>>> >
>>> >
>>> > Artifacts were signed with the 0x117C835E key which can be found in:
>>> >
>>> >
>>> >   https://downloads.apache.org/hbase/KEYS
>>> >
>>> >
>>> > To learn more about Apache hbase, please see
>>> >
>>> >
>>> >   http://hbase.apache.org/
>>> >
>>> >
>>> > Thanks,
>>> >
>>> > Your HBase Release Manager
>>>
>>

Re: [VOTE] HBase 2.4.14 release candidate (RC0) is available

[Huaxiang Sun]

Ok, I went through the jira list for 2.4.14 again. For some reason, I
marked "fixed version" for HBASE-27301 by accident, which was supposed to
be

HBASE-27125. For HBASE-27215, it was released in 2.4.13, however, its
"fixed version" is marked as 2.4.14 by accident.

So HBASE-27215 was not included in 2.4.13's release notes.


I corrected both jira's fixed versions and respining a new RC.




On Tue, Aug 23, 2022 at 8:44 AM Huaxiang Sun  wrote:

> Sorry to miss your message, checking it now, I may marked the wrong jira,
> let me check the jira list and respin RC.
>
> Thanks
> Huaxiang
>
> On Mon, Aug 22, 2022 at 6:41 PM 张铎(Duo Zhang) 
> wrote:
>
>> Sorry but -1.
>>
>> I replied on HBASE-27301 that the issue should be included in 2.4.14
>> but got no response.
>>
>> And I checked the CHANGES.md, there is no HBASE-27301 in it, neither
>> in 2.4.13 nor 2.4.14.
>>
>> Please fix this issue and publish an RC again.
>>
>> Thanks.
>>
>> Huaxiang Sun  于2022年8月23日周二 07:15写道：
>> >
>> > Please vote on this Apache hbase release candidate,
>> >
>> > hbase-2.4.14RC0
>> >
>> >
>> > The VOTE will remain open for at least 72 hours.
>> >
>> >
>> > [ ] +1 Release this package as Apache hbase 2.4.14
>> >
>> > [ ] -1 Do not release this package because ...
>> >
>> >
>> > The tag to be voted on is 2.4.14RC0:
>> >
>> >
>> >   https://github.com/apache/hbase/tree/2.4.14RC0
>> >
>> >
>> > This tag currently points to git reference
>> >
>> >
>> >   f507fb71c6724ed44eb4ac25bbdd3e35b2441a1a
>> >
>> >
>> > The release files, including signatures, digests, as well as CHANGES.md
>> >
>> > and RELEASENOTES.md included in this RC can be found at:
>> >
>> >
>> >   https://dist.apache.org/repos/dist/dev/hbase/2.4.14RC0/
>> >
>> >
>> > Maven artifacts are available in a staging repository at:
>> >
>> >
>> >
>> https://repository.apache.org/content/repositories/orgapachehbase-1493/
>> >
>> >
>> > Artifacts were signed with the 0x117C835E key which can be found in:
>> >
>> >
>> >   https://downloads.apache.org/hbase/KEYS
>> >
>> >
>> > To learn more about Apache hbase, please see
>> >
>> >
>> >   http://hbase.apache.org/
>> >
>> >
>> > Thanks,
>> >
>> > Your HBase Release Manager
>>
>

Re: [VOTE] HBase 2.4.14 release candidate (RC0) is available

[Huaxiang Sun]

Sorry to miss your message, checking it now, I may marked the wrong jira,
let me check the jira list and respin RC.

Thanks
Huaxiang

On Mon, Aug 22, 2022 at 6:41 PM 张铎(Duo Zhang)  wrote:

> Sorry but -1.
>
> I replied on HBASE-27301 that the issue should be included in 2.4.14
> but got no response.
>
> And I checked the CHANGES.md, there is no HBASE-27301 in it, neither
> in 2.4.13 nor 2.4.14.
>
> Please fix this issue and publish an RC again.
>
> Thanks.
>
> Huaxiang Sun  于2022年8月23日周二 07:15写道：
> >
> > Please vote on this Apache hbase release candidate,
> >
> > hbase-2.4.14RC0
> >
> >
> > The VOTE will remain open for at least 72 hours.
> >
> >
> > [ ] +1 Release this package as Apache hbase 2.4.14
> >
> > [ ] -1 Do not release this package because ...
> >
> >
> > The tag to be voted on is 2.4.14RC0:
> >
> >
> >   https://github.com/apache/hbase/tree/2.4.14RC0
> >
> >
> > This tag currently points to git reference
> >
> >
> >   f507fb71c6724ed44eb4ac25bbdd3e35b2441a1a
> >
> >
> > The release files, including signatures, digests, as well as CHANGES.md
> >
> > and RELEASENOTES.md included in this RC can be found at:
> >
> >
> >   https://dist.apache.org/repos/dist/dev/hbase/2.4.14RC0/
> >
> >
> > Maven artifacts are available in a staging repository at:
> >
> >
> >
> https://repository.apache.org/content/repositories/orgapachehbase-1493/
> >
> >
> > Artifacts were signed with the 0x117C835E key which can be found in:
> >
> >
> >   https://downloads.apache.org/hbase/KEYS
> >
> >
> > To learn more about Apache hbase, please see
> >
> >
> >   http://hbase.apache.org/
> >
> >
> > Thanks,
> >
> > Your HBase Release Manager
>

[VOTE] HBase 2.4.14 release candidate (RC0) is available

[Huaxiang Sun]

Please vote on this Apache hbase release candidate,

hbase-2.4.14RC0


The VOTE will remain open for at least 72 hours.


[ ] +1 Release this package as Apache hbase 2.4.14

[ ] -1 Do not release this package because ...


The tag to be voted on is 2.4.14RC0:


  https://github.com/apache/hbase/tree/2.4.14RC0


This tag currently points to git reference


  f507fb71c6724ed44eb4ac25bbdd3e35b2441a1a


The release files, including signatures, digests, as well as CHANGES.md

and RELEASENOTES.md included in this RC can be found at:


  https://dist.apache.org/repos/dist/dev/hbase/2.4.14RC0/


Maven artifacts are available in a staging repository at:


  https://repository.apache.org/content/repositories/orgapachehbase-1493/


Artifacts were signed with the 0x117C835E key which can be found in:


  https://downloads.apache.org/hbase/KEYS


To learn more about Apache hbase, please see


  http://hbase.apache.org/


Thanks,

Your HBase Release Manager

Re: Time for hbase-2.4.14 release

[Huaxiang Sun]

Thanks Duo. I am planning to spin 2.4.14RC0 on this Friday, if there is any
jira you want to include in this RC, please backport them to branch-2.4
before this Friday.

Thanks
Huaxiang

On Tue, Aug 16, 2022 at 8:11 PM 张铎(Duo Zhang)  wrote:

> Thanks for taking care of this!
>
> Huaxiang Sun  于2022年8月17日周三 02:33写道：
> >
> > Hi Folks,
> >
> > There are quite some critical fixes after 2.4.13, especially memory
> > leaks in SASL implementation and ByteBuffAllocator. Since 2.5.0 release
> is
> > being worked on, there is a rolling back issue from 2.5.0 back to 2.4.*
> > release which is fixed in the coming 2.4.14 release. 2.4.13 was released
> on
> > 7/1/22, it is about time for 2.4.14 release.
> >
> >  Andrew is on PTO for two weeks. I volunteer to run this release
> unless
> > someone else wants to do.
> >
> > Please let me know if there are any concerns.
> >
> > Best Regards,
> > Huaxiang
>

Time for hbase-2.4.14 release

[Huaxiang Sun]

Hi Folks,

There are quite some critical fixes after 2.4.13, especially memory
leaks in SASL implementation and ByteBuffAllocator. Since 2.5.0 release is
being worked on, there is a rolling back issue from 2.5.0 back to 2.4.*
release which is fixed in the coming 2.4.14 release. 2.4.13 was released on
7/1/22, it is about time for 2.4.14 release.

 Andrew is on PTO for two weeks. I volunteer to run this release unless
someone else wants to do.

Please let me know if there are any concerns.

Best Regards,
Huaxiang

[jira] [Resolved] (HBASE-27296) Some Cell's implementation of toString() such as IndividualBytesFieldCell prints out value and tags which is too verbose

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-27296.
--
Fix Version/s: 2.5.0
   3.0.0-alpha-4
   2.4.14
   Resolution: Fixed

> Some Cell's implementation of toString() such as IndividualBytesFieldCell 
> prints out value and tags which is too verbose
> 
>
> Key: HBASE-27296
> URL: https://issues.apache.org/jira/browse/HBASE-27296
> Project: HBase
>  Issue Type: Improvement
>  Components: logging
>Affects Versions: 2.4.12
>    Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Minor
> Fix For: 2.5.0, 3.0.0-alpha-4, 2.4.14
>
>
> One of users sees cells >10Mb are logged when over limit at their client log. 
> Checked the code, toString() behavior is not consistent, mostly does not 
> include values and tags. Change toString() to exclude tags/value.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27296) IndividualBytesFieldCell#toString() prints out value and tags which is too verbose.

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-27296:


 Summary: IndividualBytesFieldCell#toString() prints out value and 
tags which is too verbose.
 Key: HBASE-27296
 URL: https://issues.apache.org/jira/browse/HBASE-27296
 Project: HBase
  Issue Type: Improvement
  Components: logging
Affects Versions: 2.4.12
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun


One of users sees cells >10Mb are logged when over limit. 

Checked the code, toString() behavior is not consistent, mostly does not 
include values and tags. Change toString() to exclude tags/value.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27250) MasterRpcService#setRegionStateInMeta does not support replica region encodedNames or region names

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-27250:


 Summary: MasterRpcService#setRegionStateInMeta does not support 
replica region encodedNames or region names
 Key: HBASE-27250
 URL: https://issues.apache.org/jira/browse/HBASE-27250
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.4.13
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun


MasterRpcServices#setRegionStateInMeta does not support replica region names, 
it assumes the primary region only. This makes HBCK2's setRegionState for 
replica region fails. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27181) Replica region support in HBCK2 setRegionState option

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-27181:


 Summary: Replica region support in HBCK2 setRegionState option
 Key: HBASE-27181
 URL: https://issues.apache.org/jira/browse/HBASE-27181
 Project: HBase
  Issue Type: Improvement
  Components: hbck2
Affects Versions: 2.4.13
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun


Replica region id is  not recognized by hbck2's setRegionState as it does not 
show up in meta. We run into cases that it needs to set region state in meta 
for replica regions in order to fix inconsistency. We ended up writing the 
state manually into meta table and did a master failover to sync state from 
meta table. 

 

hbck2's setRegionState needs to support replica region id and handles it nicely.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (HBASE-27025) Change Hbase book's description for "74.7.3. Load Balancing META table load"

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-27025.
--
Fix Version/s: 3.0.0-alpha-4
   Resolution: Fixed

Merged into the master branch.

> Change Hbase book's description for "74.7.3. Load Balancing META table load"
> 
>
> Key: HBASE-27025
> URL: https://issues.apache.org/jira/browse/HBASE-27025
> Project: HBase
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 2.4.12
>Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Minor
> Fix For: 3.0.0-alpha-4
>
>
> HBASE-26618 involves primary meta region in meta scan. The description in 
> hbase book is inaccurate. Update it accordingly.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Resolved] (HBASE-26649) Support meta replica LoadBalance mode for RegionLocator#getAllRegionLocations()

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-26649.
--
Fix Version/s: 2.5.0
   3.0.0-alpha-3
   2.4.13
 Release Note: When setting 'hbase.locator.meta.replicas.mode' to 
"LoadBalance" at HBase client, RegionLocator#getAllRegionLocations() now load 
balances across all Meta Replica Regions. Please note,  results from 
non-primary meta replica regions may contain stale data. 
   Resolution: Fixed

> Support meta replica LoadBalance mode for 
> RegionLocator#getAllRegionLocations()
> ---
>
> Key: HBASE-26649
> URL: https://issues.apache.org/jira/browse/HBASE-26649
> Project: HBase
>  Issue Type: Improvement
>  Components: meta replicas
>Affects Versions: 2.4.9
>Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.13
>
>
> When HBase application restarts, its meta cache is empty. Normally, it will 
> fill the meta cache one region at a time by scanning the meta region. This 
> will cause huge pressure to the region server hosting meta during application 
> restart. 
> It can prefetching all region locations by calling 
> RegionLocator#getAllRegionLocations().Meta replica LoadBalance mode is 
> support in 2.4, it will be nice to load balance 
> RegionLocator#getAllRegionLocations() to all meta replica regions so batch 
> scan can spread across all meta replica regions.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (HBASE-27087) TestQuotaThrottle times out in branch-2.5.

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-27087:


 Summary: TestQuotaThrottle times out in branch-2.5.
 Key: HBASE-27087
 URL: https://issues.apache.org/jira/browse/HBASE-27087
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 2.5.0
Reporter: Huaxiang Sun


With branch-2.5, TestQuotaThrottle times out. Need to investigate.

 
h3. Error Message

Failed after attempts=7, exceptions: 2022-06-03T11:26:33.418Z, 
RpcRetryingCaller\{globalStartTime=2022-06-03T11:26:33.418Z, pause=250, 
maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: 
java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException: 
KeeperErrorCode = NoNode for /hbase/master 2022-06-03T11:26:33.418Z, 
RpcRetryingCaller\{globalStartTime=2022-06-03T11:26:33.418Z, pause=250, 
maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: 
java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException: 
KeeperErrorCode = NoNode for /hbase/master 2022-06-03T11:26:33.418Z, 
RpcRetryingCaller\{globalStartTime=2022-06-03T11:26:33.418Z, pause=250, 
maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: 
java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException: 
KeeperErrorCode = NoNode for /hbase/master 2022-06-03T11:26:33.418Z, 
RpcRetryingCaller\{globalStartTime=2022-06-03T11:26:33.418Z, pause=250, 
maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: 
java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException: 
KeeperErrorCode = NoNode for /hbase/master 2022-06-03T11:26:33.418Z, 
RpcRetryingCaller\{globalStartTime=2022-06-03T11:26:33.418Z, pause=250, 
maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: 
java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException: 
KeeperErrorCode = NoNode for /hbase/master 2022-06-03T11:26:33.418Z, 
RpcRetryingCaller\{globalStartTime=2022-06-03T11:26:33.418Z, pause=250, 
maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: 
java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException: 
KeeperErrorCode = NoNode for /hbase/master 2022-06-03T11:26:33.418Z, 
RpcRetryingCaller\{globalStartTime=2022-06-03T11:26:33.418Z, pause=250, 
maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: 
java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException: 
KeeperErrorCode = NoNode for /hbase/master



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Reopened] (HBASE-26962) Add mob info in web UI

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun reopened HBASE-26962:
--

The commit caused branch-2 build failure. Can you fix the build error and 
resubmit a patch? Thanks.

> Add mob info in web UI
> --
>
> Key: HBASE-26962
> URL: https://issues.apache.org/jira/browse/HBASE-26962
> Project: HBase
>  Issue Type: Improvement
>  Components: UI
>Reporter: Xuesen Liang
>Assignee: Xuesen Liang
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Add mob store info in web UI.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

Re: [DISCUSS] First release candidate for 2.5.0 (RC0) is available for evaluation

[Huaxiang Sun]

+1 to make the change everywhere. Sometimes, there is a need to update the
release field after jira is released (in theory, it should not happen).

Thanks
Huaxiang


On Thu, Jun 2, 2022 at 10:56 AM Andrew Purtell  wrote:

> Even for branch-2.4, we can start placing links into the file, at the top,
> leaving the output of the prior process in place below. I think it is fine
> to make this change everywhere. Curious what others think.
>
> On Thu, Jun 2, 2022 at 10:54 AM Andrew Purtell 
> wrote:
>
> > I see, that would be totally fine with me if we just do the link instead
> > of generating the files every time.
> >
> > On Thu, Jun 2, 2022 at 9:31 AM 张铎(Duo Zhang) 
> > wrote:
> >
> >> Oh, wait a minute. IIRC we have a consensus to use the jira page as
> >> release
> >> note instead of committing a file to the repo? The problem is that,
> >> sometimes we just missed something in CHANGES.md or RELEASENOTES.md, no
> >> actually code change, but since we have committed these files to our git
> >> repo, we have to sink the RC and generate a new one.
> >>
> >> IIRC for making the two alpha releases for 3.0.0, I have already started
> >> to
> >> link the jira page instead of CHANGES.md and RELEASENOTES.md. Maybe we
> >> could just remove these steps from the create release scripts? But maybe
> >> for branch-2.4 we still need to use these steps...
> >>
> >> Thanks.
> >>
> >> Andrew Purtell  于2022年6月2日周四 22:56写道：
> >>
> >> > I just noticed the release notes issue on the branch last evening.
> Let’s
> >> > add the spotless application to the release note generation step in
> >> > create-release at least or it will keep happening. The RM does not
> get a
> >> > chance to intervene. The script uses the RM credentials to commit
> >> > immediately after generating the files.
> >> >
> >> > > On Jun 2, 2022, at 5:31 AM, 张铎  wrote:
> >> > >
> >> > > Oh, I think we should include HBASE-27023, it fixes the license
> issue
> >> > after
> >> > > we apply the spotless formatter where it will add the license header
> >> > > automatically.
> >> > >
> >> > > And also, it seems that the generated RELEASENOTE.md will introduce
> >> some
> >> > > lines ending with whitespace, then it will cause pre commit build
> >> > failure.
> >> > >
> >> > > I think we should run a spotless:apply before committing the files
> in
> >> the
> >> > > release scripts, and maybe we should also add spotless:check in the
> >> mvn
> >> > > verify stage so it will also fail the mvn install command, which
> will
> >> let
> >> > > developers pay more attention on it.
> >> > >
> >> > > Let me open an issue to land these changes.
> >> > >
> >> > > 张铎(Duo Zhang)  于2022年6月2日周四 11:14写道：
> >> > >
> >> > >> OK, got it. Thanks for the clarification~
> >> > >>
> >> > >> Andrew Purtell  于2022年6月2日周四 11:12写道：
> >> > >>
> >> > >>> I don’t think a flaky test is cause to fail the release. So these
> >> all
> >> > >>> have a fix version of 2.5.1. I mentioned them so people could be
> >> aware
> >> > of
> >> > >>> these known issue when evaluating their own findings. Hope that
> >> helps.
> >> > >>>
> >> > > On Jun 1, 2022, at 8:11 PM, 张铎  wrote:
> >> > >
> >> > > On flakyness, if it is a big problem on promoting the RC, I
> >> think we
> >> > >>> can
> >> >  ignore the test first(instead of removing it).
> >> > 
> >> >  Can open a follow on issue to make it stable.
> >> > 
> >> >  Thanks.
> >> > 
> >> >  Andrew Purtell  于2022年6月1日周三 23:25写道：
> >> > 
> >> > > Makes sense and if nobody has a concern we can do that.
> >> > >
> >> > > Also you wanted to support the change to the RSGroup API and put
> >> back
> >> > >>> the
> >> > > unit test (and fix it to not flake), correct, which seems fine
> to
> >> me
> >> > >>> too.
> >> > >
> >> > >> On Tue, May 31, 2022 at 10:54 PM 张铎(Duo Zhang) <
> >> > palomino...@gmail.com
> >> > 
> >> > >> wrote:
> >> > >>
> >> > >> The ReplicationLoadSink change is done in HBASE-26490, this is
> >> the
> >> > >>> commit
> >> > >>
> >> > >>
> >> > >>
> >> > >
> >> > >>>
> >> >
> >>
> https://github.com/apache/hbase/commit/b5b286d793f00ec8b9ed02f51fd9324e46f29c86
> >> > >>
> >> > >> The constructor is marked as IA.Private so I do not think we
> >> need to
> >> > >> restore it back.
> >> > >>
> >> > >> Thanks.
> >> > >>
> >> > >> Andrew Purtell  于2022年6月1日周三 13:08写道：
> >> > >>
> >> > >>> This is not a VOTE.
> >> > >>>
> >> > >>> HBase 2.5.0RC0 is ready for testing, other evaluation, and
> >> > > consideration
> >> > >> of
> >> > >>> compatibility concerns.
> >> > >>>
> >> > >>> The release notes need to be rebased on the latest 2.4.x
> >> release,
> >> > > 2.4.12.
> >> > >>> Fix
> >> > >>> versions on relvant JIRAs must be cleaned up first. This is in
> >> > > progress.
> >> > >> I
> >> > >>> also plan to try our new audit tool for matching git history
> and
> >> > 

Re: [NOTICE] Please be careful when merging a PR with failure pre commit result

[Huaxiang Sun]

Terribly sorry here! I did not notice Duo's reply in the PR and did not get
chance to fix the error in time.
Will pay more attention and be more careful when merging PR next time.

Thanks again Duo for fixing the error.

Huaxiang

On Wed, May 25, 2022 at 6:09 AM 张铎(Duo Zhang)  wrote:

> Since our UTs are not stable enough, sometimes when we can make sure that
> the failure UTs are not caused by the PR, we will merge the PR with a big
> red cross.
>
> But recently, I've seen several times our committers just merged a PR with
> a big red cross on the spotless check. It is very easy to fix, just run a
> 'mvn spotless:apply'. But if you merge it without fixing the issue, it will
> confuse all later contributors because it is not something like the flaky
> UTs, it will always fail and cause the pre commit check result for all the
> later PRs to fail.
>
> The newest one which breaks the build is
> https://github.com/apache/hbase/pull/4442. I've replied on the PR but got
> no response yet. This time I will commit an addendum to fix it, but I hope
> all our committers can take the responsibility in the future.
>
> Thanks. Regards
>

[jira] [Created] (HBASE-27025) Change Hbase book's description for "74.7.3. Load Balancing META table load"

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-27025:


 Summary: Change Hbase book's description for "74.7.3. Load 
Balancing META table load"
 Key: HBASE-27025
 URL: https://issues.apache.org/jira/browse/HBASE-27025
 Project: HBase
  Issue Type: Improvement
  Components: documentation
Affects Versions: 2.4.12
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun


HBASE-26618 involves primary meta region in meta scan. The description in hbase 
book is inaccurate. Update it accordingly.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

Re: [VOTE] First release candidate for HBase 2.4.12 (RC0) is available

[Huaxiang Sun]

More testing data with openjdk version "11.0.6"

* Signature: ok

* Checksum : ok

* Rat check (11.0.6): ok

 - mvn clean apache-rat:check -D hadoop.profile=3.0

* Built from source (11.0.6): ok

 - mvn clean install -D hadoop.profile=3.0 -DskipTests

* Unit tests pass (11.0.6): ok

 - mvn package -P runSmallTests -D hadoop.profile=3.0
-Dsurefire.rerunFailingTestsCount=3

I started an ITBLL chaos run with meta replica LoadBalance mode enabled at
client side and server side configured with meta asyncWal replication, 13
node k8s cluster.  Inserted 250M rows and it is verified correctly.


On Thu, May 5, 2022 at 7:28 AM Huaxiang Sun  wrote:

> +1
>
> * Signature: ok
>
> * Checksum : ok
>
> * Rat check (1.8.0_242): ok
>
>  - mvn clean apache-rat:check
>
> * Built from source (1.8.0_242): ok
>
>  - mvn clean install  -DskipTests
>
> * Unit tests pass (1.8.0_242): ok
>
>  - mvn package -P runSmallTests
> -Dsurefire.rerunFailingTestsCount=3
>
>
>
>
> On Thu, May 5, 2022 at 2:57 AM Xiaolin Ha  wrote:
>
>> +1
>>
>> * Signature: ok
>> * Checksum: ok
>> * Rat check(1.8.0_202): ok
>>- mvn clean apache-rat:check
>> * Built from source(1.8.0_202): ok
>>- mvn clean install -DskipTests
>> * Unit tests passed: failed
>>- mvn package -P runAllTests -Dsurefire.rerunFailingTestsCount=3
>>
>> Checked CHANGES and RELEASENOTES
>>
>> Standalone mode:
>> * Basic shell commands
>> * Checked Master and RS UI
>> * LLT 1M rows(PE)
>>
>

[jira] [Resolved] (HBASE-26984) Chaos Monkey thread dies in ITBLL Chaos GracefulRollingRestartRsAction

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-26984.
--
Fix Version/s: 2.5.0
   3.0.0-alpha-3
   Resolution: Fixed

> Chaos Monkey thread dies in ITBLL Chaos GracefulRollingRestartRsAction 
> ---
>
> Key: HBASE-26984
> URL: https://issues.apache.org/jira/browse/HBASE-26984
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Affects Versions: 2.4.11
>    Reporter: Huaxiang Sun
>    Assignee: Huaxiang Sun
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> Run itbll chaos monkey in k8s cluster, found chaos monkey thread died in 
> GracefulRollingRestartRsAction. 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

Re: [VOTE] First release candidate for HBase 2.4.12 (RC0) is available

[Huaxiang Sun]

+1

* Signature: ok

* Checksum : ok

* Rat check (1.8.0_242): ok

 - mvn clean apache-rat:check

* Built from source (1.8.0_242): ok

 - mvn clean install  -DskipTests

* Unit tests pass (1.8.0_242): ok

 - mvn package -P runSmallTests  -Dsurefire.rerunFailingTestsCount=3




On Thu, May 5, 2022 at 2:57 AM Xiaolin Ha  wrote:

> +1
>
> * Signature: ok
> * Checksum: ok
> * Rat check(1.8.0_202): ok
>- mvn clean apache-rat:check
> * Built from source(1.8.0_202): ok
>- mvn clean install -DskipTests
> * Unit tests passed: failed
>- mvn package -P runAllTests -Dsurefire.rerunFailingTestsCount=3
>
> Checked CHANGES and RELEASENOTES
>
> Standalone mode:
> * Basic shell commands
> * Checked Master and RS UI
> * LLT 1M rows(PE)
>

[jira] [Created] (HBASE-26984) Chaos Monkey thread dies in ITBLL Chaos GracefulRollingRestartRsAction

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-26984:


 Summary: Chaos Monkey thread dies in ITBLL Chaos 
GracefulRollingRestartRsAction 
 Key: HBASE-26984
 URL: https://issues.apache.org/jira/browse/HBASE-26984
 Project: HBase
  Issue Type: Bug
  Components: integration tests
Affects Versions: 2.4.11
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun


Run itbll chaos monkey in k8s cluster, found chaos monkey thread died in 
GracefulRollingRestartRsAction. 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Resolved] (HBASE-26618) Involving primary meta region in meta scan with CatalogReplicaLoadBalanceSimpleSelector

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-26618.
--
Fix Version/s: 2.5.0
   3.0.0-alpha-3
   2.4.12
 Release Note: When META replica LoadBalance mode is enabled at 
client-side, clients will try to read from one META region first. If META 
location is from any non-primary META regions, in case of errors, it will fall 
back to the primary META region.
   Resolution: Fixed

> Involving primary meta region in meta scan with 
> CatalogReplicaLoadBalanceSimpleSelector
> ---
>
> Key: HBASE-26618
> URL: https://issues.apache.org/jira/browse/HBASE-26618
> Project: HBase
>  Issue Type: Improvement
>  Components: meta replicas
>Affects Versions: 2.4.9
>    Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Minor
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.12
>
>
> In the current release with Meta replica LoadBalance mode, the primary meta 
> region is not serving the meta scan (only meta replica region serves the 
> read). When the result from meta replica region is stale, it will go to 
> primary meta region for up-to-date location. 
> From our experience, the primary meta region serves very less read traffic, 
> so it will be better to load balance read traffic across the primary meta 
> region as well.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: [ANNOUNCE] New HBase committer Bryan Beaudreault

[Huaxiang Sun]

Congratulations, Bryan!

Sent from my iPhone

> On Apr 11, 2022, at 2:30 AM, Jan Hentschel  
> wrote:
> 
> Congratulations and welcome!
> 
> From: 张铎(Duo Zhang) 
> Date: Saturday, April 9, 2022 at 1:45 PM
> To: HBase Dev List , hbase-user 
> , user-zh 
> Subject: [ANNOUNCE] New HBase committer Bryan Beaudreault
> [CAUTION] The sender of this email is outside our organization. Please DO NOT 
> CLICK links, download attachments or respond unless you recognize the sender 
> and know the content is safe. Please contact IT immediately in case you find 
> it suspicious.
> 
> On behalf of the Apache HBase PMC, I am pleased to announce that Bryan
> Beaudreault(bbeaudreault) has accepted the PMC's invitation to become a
> committer on the project. We appreciate all of Bryan's generous
> contributions thus far and look forward to his continued involvement.
> 
> Congratulations and welcome, Bryan Beaudreault!
> 
> 我很高兴代表 Apache HBase PMC 宣布 Bryan Beaudreault 已接受我们的邀请，成为 Apache HBase 项目的
> Committer。感谢 Bryan Beaudreault 一直以来为 HBase 项目做出的贡献，并期待他在未来继续承担更多的责任。
> 
> 欢迎 Bryan Beaudreault！

Re: [ANNOUNCE] Please welcome Xiaolin Ha(哈晓琳) to the HBase PMC

[Huaxiang Sun]

Congratulations, Xiaolin!

Sent from my iPhone

> On Apr 11, 2022, at 2:30 AM, Jan Hentschel  
> wrote:
> 
> Congratulations and welcome!
> 
> From: 张铎(Duo Zhang) 
> Date: Saturday, April 9, 2022 at 3:11 PM
> To: HBase Dev List , hbase-user 
> , user-zh 
> Subject: [ANNOUNCE] Please welcome Xiaolin Ha(哈晓琳) to the HBase PMC
> [CAUTION] The sender of this email is outside our organization. Please DO NOT 
> CLICK links, download attachments or respond unless you recognize the sender 
> and know the content is safe. Please contact IT immediately in case you find 
> it suspicious.
> 
> On behalf of the Apache HBase PMC I am pleased to announce that Xiaolin Ha
> has accepted our invitation to become a PMC member on the Apache HBase
> project. We appreciate Xiaolin Ha stepping up to take more responsibility in
> the HBase project.
> 
> Please join me in welcoming Xiaolin Ha to the HBase PMC!
> 
> 我很高兴代表 Apache HBase PMC 宣布哈晓琳已接受我们的邀请，
> 成为 Apache HBase 项目的 PMC 成员。感谢哈晓琳愿意在 HBase 项目
> 中承担更大的责任。
> 
> 欢迎哈晓琳！

Re: Meta replicas in LoadBalance mode broken in 2.4?

[Huaxiang Sun]

Sorry for replying late. Yeah, at server side, since region move is very rare 
in our production cluster, we did not enable the async wal replication. It is 
only being tested in itbll cluster.

On 2022/03/31 13:35:01 Andor Molnar wrote:
> One more question Huaxiang which makes me confused:
> 
> You mentioned that "At server side, we use hfile refresh instead of wal 
> replication.” which means that async wal replication for meta is not tested 
> by you.
> 
> Is that correct?
> 
> Andor
> 
> 
> 
> 
> > On 2022. Mar 30., at 19:22, Andor Molnar  wrote:
> > 
> > Hi Huaxiang,
> > 
> > Given that you already use this feature in production with no problems, I 
> > created a short patch to remove the “be careful” warnings from the HBase 
> > documentation.
> > 
> > PTAL if you agree.
> > 
> > https://github.com/apache/hbase/pull/4301
> > 
> > Thanks,
> > Andor
> > 
> > 
> > 
> > 
> >> On 2022. Mar 29., at 18:52, Huaxiang Sun  wrote:
> >> 
> >> This is great, thanks for the testing results!
> >> 
> >> Huaxiang
> >> 
> >> On 2022/03/29 13:29:48 Andor Molnar wrote:
> >>> Works!
> >>> 
> >>> I enabled async wal replication with the suggested option and ITBLL ran 
> >>> successfully.
> >>> 
> >>> generator step: 
> >>> hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList generator 
> >>> 15 100 /tmp/hbase-itbll
> >>> 
> >>> verification step:
> >>> hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList verify 
> >>> /tmp/hbase-itbll-verify 15
> >>> 
> >>> Both succeeded. I also confirmed that meta replicas are written and read 
> >>> by the clients, so must be in load balance mode.
> >>> 
> >>> Thanks for the help!
> >>> 
> >>> Andor
> >>> 
> >>> 
> >>> 
> >>>> On 2022. Mar 27., at 6:59, Huaxiang Sun  wrote:
> >>>> 
> >>>> It makes sense to turn on async wal replication when 
> >>>> hbase.meta.replicas.use = true. Let me run couple rounds of itbll with 
> >>>> hbase.region.replica.replication.catalog.enabled (lastest 2.4 and 2.5.0 
> >>>> candidates) to get more confidence before proposing turn on async wal 
> >>>> replication for meta.
> >>>> 
> >>>> Thanks,
> >>>> Huaxiang
> >>>> 
> >>>> On 2022/03/26 04:03:15 Andrew Purtell wrote:
> >>>>> Just to be clear when I say "it seems pointless to have meta replicas 
> >>>>> which
> >>>>> do not actually receive updates (by default)", what I should have said 
> >>>>> is
> >>>>> 'timely updates', because a long delay in updating meta might as well 
> >>>>> be a
> >>>>> missed update.
> >>>>> 
> >>>>> On Fri, Mar 25, 2022 at 9:01 PM Andrew Purtell  
> >>>>> wrote:
> >>>>> 
> >>>>>>> "Async WAL replication for META is added as a new feature in 2.4.0. It
> >>>>>> is still under active development. Use with caution. Set
> >>>>>> hbase.region.replica.replication.catalog.enabled to enable async WAL
> >>>>>> Replication for META region replicas. It is off by default."
> >>>>>> 
> >>>>>> Do we still need this warning?
> >>>>>> 
> >>>>>> Should hbase.region.replica.replication.catalog.enabled have a default 
> >>>>>> of
> >>>>>> 'true' (enabled) if hbase.meta.replicas.use = true ? Otherwise, it 
> >>>>>> seems
> >>>>>> pointless to have meta replicas which do not actually receive updates 
> >>>>>> (by
> >>>>>> default).
> >>>>>> 
> >>>>>> 
> >>>>>> On Fri, Mar 25, 2022 at 10:51 AM Huaxiang Sun 
> >>>>>> wrote:
> >>>>>> 
> >>>>>>> Hi Andor,
> >>>>>>> 
> >>>>>>> I get what you are saying. The HFile refreshing is the old way for
> >>>>>>> replica regions to refresh hfiles periodically, default is 5 minutes. 
> >>>>>>> In
> &g

Re: Meta replicas in LoadBalance mode broken in 2.4?

[Huaxiang Sun]

This is great, thanks for the testing results!

Huaxiang

On 2022/03/29 13:29:48 Andor Molnar wrote:
> Works!
> 
> I enabled async wal replication with the suggested option and ITBLL ran 
> successfully.
> 
> generator step: 
> hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList generator 15 
> 100 /tmp/hbase-itbll
> 
> verification step:
> hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList verify 
> /tmp/hbase-itbll-verify 15
> 
> Both succeeded. I also confirmed that meta replicas are written and read by 
> the clients, so must be in load balance mode.
> 
> Thanks for the help!
> 
> Andor
> 
> 
> 
> > On 2022. Mar 27., at 6:59, Huaxiang Sun  wrote:
> > 
> > It makes sense to turn on async wal replication when 
> > hbase.meta.replicas.use = true. Let me run couple rounds of itbll with 
> > hbase.region.replica.replication.catalog.enabled (lastest 2.4 and 2.5.0 
> > candidates) to get more confidence before proposing turn on async wal 
> > replication for meta.
> > 
> > Thanks,
> > Huaxiang
> > 
> > On 2022/03/26 04:03:15 Andrew Purtell wrote:
> >> Just to be clear when I say "it seems pointless to have meta replicas which
> >> do not actually receive updates (by default)", what I should have said is
> >> 'timely updates', because a long delay in updating meta might as well be a
> >> missed update.
> >> 
> >> On Fri, Mar 25, 2022 at 9:01 PM Andrew Purtell  wrote:
> >> 
> >>>> "Async WAL replication for META is added as a new feature in 2.4.0. It
> >>> is still under active development. Use with caution. Set
> >>> hbase.region.replica.replication.catalog.enabled to enable async WAL
> >>> Replication for META region replicas. It is off by default."
> >>> 
> >>> Do we still need this warning?
> >>> 
> >>> Should hbase.region.replica.replication.catalog.enabled have a default of
> >>> 'true' (enabled) if hbase.meta.replicas.use = true ? Otherwise, it seems
> >>> pointless to have meta replicas which do not actually receive updates (by
> >>> default).
> >>> 
> >>> 
> >>> On Fri, Mar 25, 2022 at 10:51 AM Huaxiang Sun 
> >>> wrote:
> >>> 
> >>>> Hi Andor,
> >>>> 
> >>>>   I get what you are saying. The HFile refreshing is the old way for
> >>>> replica regions to refresh hfiles periodically, default is 5 minutes. In
> >>>> this itbll case, we need to have the wal replication enabled for meta
> >>>> replica. Please check out,
> >>>> 
> >>>> https://hbase.apache.org/book.html#_async_wal_replication_for_meta_table_as_of_hbase_2_4_0.
> >>>> Basically, you need to set
> >>>> "hbase.region.replica.replication.catalog.enabled" to true in the
> >>>> configuration and rerun itbll. Otherwise, all meta changes at the primary
> >>>> meta region wont be updated at the replica meta regions and it will 
> >>>> result
> >>>> in itbll failures.
> >>>> 
> >>>>   Hope this helps,
> >>>> 
> >>>>   Huaxiang
> >>>> 
> >>>> 
> >>>> On 2022/03/25 13:46:42 Andor Molnar wrote:
> >>>>> Hi Huaxiang,
> >>>>> 
> >>>>> We use 2.4.6 for the tests.
> >>>>> 
> >>>>> I run itbll with the following command:
> >>>>> 
> >>>>> hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList
> >>>> generator 15 100 /tmp/hbase-itbll
> >>>>> 
> >>>>> for the generator step and essentially jobs have failed. We can see the
> >>>> meta request are spanning out to replicas, but writes start failing after
> >>>> this due to the stale cache which is not getting updated.
> >>>>> 
> >>>>> Would you please tell me more about ‘hfile refresh’ and how to
> >>>> configure it?
> >>>>> 
> >>>>> Thanks,
> >>>>> Andor
> >>>>> 
> >>>>> 
> >>>>> 
> >>>>> 
> >>>>>> On 2022. Mar 24., at 17:43, Huaxiang Sun 
> >>>> wrote:
> >>>>>> 
> >>>>>> Hi Andor,
> >>>>>> 
> >>>>>>  Which 2.4 release do you test in your lab? We use this feature at
> >>>> production cluster with 2.4.5.
> >>>>>> At server side, we use hfile refresh instead of wal replication. I
> >>>> used to run itbll for each release with this feature enabled. How did you
> >>>> find the errors, did itbll fail?
> >>>>>> 
> >>>>>>  Regards,
> >>>>>>  Huaxiang
> >>>>> 
> >>>>> 
> >>>> 
> >>> 
> >>> 
> >>> --
> >>> Best regards,
> >>> Andrew
> >>> 
> >>> Unrest, ignorance distilled, nihilistic imbeciles -
> >>>It's what we’ve earned
> >>> Welcome, apocalypse, what’s taken you so long?
> >>> Bring us the fitting end that we’ve been counting on
> >>>   - A23, Welcome, Apocalypse
> >>> 
> >> 
> >> 
> >> -- 
> >> Best regards,
> >> Andrew
> >> 
> >> Unrest, ignorance distilled, nihilistic imbeciles -
> >>It's what we’ve earned
> >> Welcome, apocalypse, what’s taken you so long?
> >> Bring us the fitting end that we’ve been counting on
> >>   - A23, Welcome, Apocalypse
> >> 
> 
> 

Re: Meta replicas in LoadBalance mode broken in 2.4?

[Huaxiang Sun]

I will start to work on it.

Thanks
Huaxiang

On 2022/03/27 18:42:06 Andrew Purtell wrote:
> On Sat, Mar 26, 2022 at 10:00 PM Huaxiang Sun 
> wrote:
> 
> > It makes sense to turn on async wal replication when
> > hbase.meta.replicas.use = true. Let me run couple rounds of itbll with
> > hbase.region.replica.replication.catalog.enabled (lastest 2.4 and 2.5.0
> > candidates) to get more confidence before proposing turn on async wal
> > replication for meta.
> >
> 
> +1
> 
> Thanks,
> > Huaxiang
> >
> > On 2022/03/26 04:03:15 Andrew Purtell wrote:
> > > Just to be clear when I say "it seems pointless to have meta replicas
> > which
> > > do not actually receive updates (by default)", what I should have said is
> > > 'timely updates', because a long delay in updating meta might as well be
> > a
> > > missed update.
> > >
> > > On Fri, Mar 25, 2022 at 9:01 PM Andrew Purtell 
> > wrote:
> > >
> > > > > "Async WAL replication for META is added as a new feature in 2.4.0.
> > It
> > > > is still under active development. Use with caution. Set
> > > > hbase.region.replica.replication.catalog.enabled to enable async WAL
> > > > Replication for META region replicas. It is off by default."
> > > >
> > > > Do we still need this warning?
> > > >
> > > > Should hbase.region.replica.replication.catalog.enabled have a default
> > of
> > > > 'true' (enabled) if hbase.meta.replicas.use = true ? Otherwise, it
> > seems
> > > > pointless to have meta replicas which do not actually receive updates
> > (by
> > > > default).
> > > >
> > > >
> > > > On Fri, Mar 25, 2022 at 10:51 AM Huaxiang Sun 
> > > > wrote:
> > > >
> > > >> Hi Andor,
> > > >>
> > > >>I get what you are saying. The HFile refreshing is the old way for
> > > >> replica regions to refresh hfiles periodically, default is 5 minutes.
> > In
> > > >> this itbll case, we need to have the wal replication enabled for meta
> > > >> replica. Please check out,
> > > >>
> > > >>
> > https://hbase.apache.org/book.html#_async_wal_replication_for_meta_table_as_of_hbase_2_4_0
> > .
> > > >> Basically, you need to set
> > > >> "hbase.region.replica.replication.catalog.enabled" to true in the
> > > >> configuration and rerun itbll. Otherwise, all meta changes at the
> > primary
> > > >> meta region wont be updated at the replica meta regions and it will
> > result
> > > >> in itbll failures.
> > > >>
> > > >>Hope this helps,
> > > >>
> > > >>Huaxiang
> > > >>
> > > >>
> > > >> On 2022/03/25 13:46:42 Andor Molnar wrote:
> > > >> > Hi Huaxiang,
> > > >> >
> > > >> > We use 2.4.6 for the tests.
> > > >> >
> > > >> > I run itbll with the following command:
> > > >> >
> > > >> > hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList
> > > >> generator 15 100 /tmp/hbase-itbll
> > > >> >
> > > >> > for the generator step and essentially jobs have failed. We can see
> > the
> > > >> meta request are spanning out to replicas, but writes start failing
> > after
> > > >> this due to the stale cache which is not getting updated.
> > > >> >
> > > >> > Would you please tell me more about ‘hfile refresh’ and how to
> > > >> configure it?
> > > >> >
> > > >> > Thanks,
> > > >> > Andor
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> > > On 2022. Mar 24., at 17:43, Huaxiang Sun 
> > > >> wrote:
> > > >> > >
> > > >> > > Hi Andor,
> > > >> > >
> > > >> > >   Which 2.4 release do you test in your lab? We use this feature
> > at
> > > >> production cluster with 2.4.5.
> > > >> > > At server side, we use hfile refresh instead of wal replication. I
> > > >> used to run itbll for each release with this feature enabled. How did
> > you
> > > >> find the errors, did itbll fail?
> > > >> > >
> > > >> > >   Regards,
> > > >> > >   Huaxiang
> > > >> >
> > > >> >
> > > >>
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Andrew
> > > >
> > > > Unrest, ignorance distilled, nihilistic imbeciles -
> > > > It's what we’ve earned
> > > > Welcome, apocalypse, what’s taken you so long?
> > > > Bring us the fitting end that we’ve been counting on
> > > >- A23, Welcome, Apocalypse
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew
> > >
> > > Unrest, ignorance distilled, nihilistic imbeciles -
> > > It's what we’ve earned
> > > Welcome, apocalypse, what’s taken you so long?
> > > Bring us the fitting end that we’ve been counting on
> > >- A23, Welcome, Apocalypse
> > >
> >
> 
> 
> -- 
> Best regards,
> Andrew
> 
> Unrest, ignorance distilled, nihilistic imbeciles -
> It's what we’ve earned
> Welcome, apocalypse, what’s taken you so long?
> Bring us the fitting end that we’ve been counting on
>- A23, Welcome, Apocalypse
> 

Re: Meta replicas in LoadBalance mode broken in 2.4?

[Huaxiang Sun]

Hi Andor,

Please keep us updated how itbll goes with the new server side config. Your 
result will be used a datapoint as well.

   Thanks,
   Huaxiang

On 2022/03/25 17:51:49 Huaxiang Sun wrote:
> Hi Andor,
> 
>I get what you are saying. The HFile refreshing is the old way for replica 
> regions to refresh hfiles periodically, default is 5 minutes. In this itbll 
> case, we need to have the wal replication enabled for meta replica. Please 
> check out,
> https://hbase.apache.org/book.html#_async_wal_replication_for_meta_table_as_of_hbase_2_4_0.
>  Basically, you need to set 
> "hbase.region.replica.replication.catalog.enabled" to true in the 
> configuration and rerun itbll. Otherwise, all meta changes at the primary 
> meta region wont be updated at the replica meta regions and it will result in 
> itbll failures.
> 
>Hope this helps,
> 
>Huaxiang
> 
> 
> On 2022/03/25 13:46:42 Andor Molnar wrote:
> > Hi Huaxiang,
> > 
> > We use 2.4.6 for the tests.
> > 
> > I run itbll with the following command:
> > 
> > hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList generator 
> > 15 100 /tmp/hbase-itbll
> > 
> > for the generator step and essentially jobs have failed. We can see the 
> > meta request are spanning out to replicas, but writes start failing after 
> > this due to the stale cache which is not getting updated.
> > 
> > Would you please tell me more about ‘hfile refresh’ and how to configure it?
> > 
> > Thanks,
> > Andor
> > 
> > 
> > 
> > 
> > > On 2022. Mar 24., at 17:43, Huaxiang Sun  wrote:
> > > 
> > > Hi Andor,
> > > 
> > >   Which 2.4 release do you test in your lab? We use this feature at 
> > > production cluster with 2.4.5. 
> > > At server side, we use hfile refresh instead of wal replication. I used 
> > > to run itbll for each release with this feature enabled. How did you find 
> > > the errors, did itbll fail?
> > > 
> > >   Regards,
> > >   Huaxiang
> > 
> > 
> 

Re: Meta replicas in LoadBalance mode broken in 2.4?

[Huaxiang Sun]

It makes sense to turn on async wal replication when hbase.meta.replicas.use = 
true. Let me run couple rounds of itbll with 
hbase.region.replica.replication.catalog.enabled (lastest 2.4 and 2.5.0 
candidates) to get more confidence before proposing turn on async wal 
replication for meta.

Thanks,
Huaxiang

On 2022/03/26 04:03:15 Andrew Purtell wrote:
> Just to be clear when I say "it seems pointless to have meta replicas which
> do not actually receive updates (by default)", what I should have said is
> 'timely updates', because a long delay in updating meta might as well be a
> missed update.
> 
> On Fri, Mar 25, 2022 at 9:01 PM Andrew Purtell  wrote:
> 
> > > "Async WAL replication for META is added as a new feature in 2.4.0. It
> > is still under active development. Use with caution. Set
> > hbase.region.replica.replication.catalog.enabled to enable async WAL
> > Replication for META region replicas. It is off by default."
> >
> > Do we still need this warning?
> >
> > Should hbase.region.replica.replication.catalog.enabled have a default of
> > 'true' (enabled) if hbase.meta.replicas.use = true ? Otherwise, it seems
> > pointless to have meta replicas which do not actually receive updates (by
> > default).
> >
> >
> > On Fri, Mar 25, 2022 at 10:51 AM Huaxiang Sun 
> > wrote:
> >
> >> Hi Andor,
> >>
> >>I get what you are saying. The HFile refreshing is the old way for
> >> replica regions to refresh hfiles periodically, default is 5 minutes. In
> >> this itbll case, we need to have the wal replication enabled for meta
> >> replica. Please check out,
> >>
> >> https://hbase.apache.org/book.html#_async_wal_replication_for_meta_table_as_of_hbase_2_4_0.
> >> Basically, you need to set
> >> "hbase.region.replica.replication.catalog.enabled" to true in the
> >> configuration and rerun itbll. Otherwise, all meta changes at the primary
> >> meta region wont be updated at the replica meta regions and it will result
> >> in itbll failures.
> >>
> >>Hope this helps,
> >>
> >>Huaxiang
> >>
> >>
> >> On 2022/03/25 13:46:42 Andor Molnar wrote:
> >> > Hi Huaxiang,
> >> >
> >> > We use 2.4.6 for the tests.
> >> >
> >> > I run itbll with the following command:
> >> >
> >> > hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList
> >> generator 15 100 /tmp/hbase-itbll
> >> >
> >> > for the generator step and essentially jobs have failed. We can see the
> >> meta request are spanning out to replicas, but writes start failing after
> >> this due to the stale cache which is not getting updated.
> >> >
> >> > Would you please tell me more about ‘hfile refresh’ and how to
> >> configure it?
> >> >
> >> > Thanks,
> >> > Andor
> >> >
> >> >
> >> >
> >> >
> >> > > On 2022. Mar 24., at 17:43, Huaxiang Sun 
> >> wrote:
> >> > >
> >> > > Hi Andor,
> >> > >
> >> > >   Which 2.4 release do you test in your lab? We use this feature at
> >> production cluster with 2.4.5.
> >> > > At server side, we use hfile refresh instead of wal replication. I
> >> used to run itbll for each release with this feature enabled. How did you
> >> find the errors, did itbll fail?
> >> > >
> >> > >   Regards,
> >> > >   Huaxiang
> >> >
> >> >
> >>
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Unrest, ignorance distilled, nihilistic imbeciles -
> > It's what we’ve earned
> > Welcome, apocalypse, what’s taken you so long?
> > Bring us the fitting end that we’ve been counting on
> >- A23, Welcome, Apocalypse
> >
> 
> 
> -- 
> Best regards,
> Andrew
> 
> Unrest, ignorance distilled, nihilistic imbeciles -
> It's what we’ve earned
> Welcome, apocalypse, what’s taken you so long?
> Bring us the fitting end that we’ve been counting on
>- A23, Welcome, Apocalypse
> 

Re: [DISCUSS] Use spotless to auto format pom and java code

[Huaxiang Sun]

+1, a very nice and helpful feature.



On 2022/03/25 04:04:31 "张铎(Duo Zhang)" wrote:
> Revive.
> 
> Will wait for another couple of days, if no big objections, I will move
> forward to integrate spotless into our active branches.
> 
> Thanks.
> 
> 张铎(Duo Zhang)  于2022年3月15日周二 21:17写道：
> 
> > I've filed HBASE-26617 a while ago and recently I implemented a PR for
> > master branch.
> >
> > https://github.com/apache/hbase/pull/4214
> >
> > The PR is a bit large because it will also format the pom, the actual
> > changes are here
> >
> >
> > https://github.com/apache/hbase/pull/4214/files#diff-9c5fb3d1b7e3b0f54bc5c4182965c4fe1f9023d449017cece3005d3f90e8e4d8R2673
> >
> > In the PR I make use of the eclipse formatter and import order file to
> > format our java file. For pom, I just use most of the default formatter,
> > the only exception is to expand empty element.
> >
> > The spotless plugin support setting a ratchetFrom option, which could be a
> > commit or a tag. Only files changed after this commit will be formatted, so
> > we can avoid generating a very big patch when running spotless:apply.
> >
> > So after we get this in, before submitting a PR, you can just type mvn
> > spotless:apply, then most of the checkstyle issues will be solved
> > automatically. And we could also add a spotless:check step in our pre
> > commit job, to make sure we run spotless:apply before submitting a PR.
> >
> > Just tell me what you think about the above plan. Suggestions are always
> > welcomed.
> >
> > Thanks.
> >
> 

Re: Meta replicas in LoadBalance mode broken in 2.4?

[Huaxiang Sun]

Hi Andor,

   I get what you are saying. The HFile refreshing is the old way for replica 
regions to refresh hfiles periodically, default is 5 minutes. In this itbll 
case, we need to have the wal replication enabled for meta replica. Please 
check out,
https://hbase.apache.org/book.html#_async_wal_replication_for_meta_table_as_of_hbase_2_4_0.
 Basically, you need to set "hbase.region.replica.replication.catalog.enabled" 
to true in the configuration and rerun itbll. Otherwise, all meta changes at 
the primary meta region wont be updated at the replica meta regions and it will 
result in itbll failures.

   Hope this helps,

   Huaxiang


On 2022/03/25 13:46:42 Andor Molnar wrote:
> Hi Huaxiang,
> 
> We use 2.4.6 for the tests.
> 
> I run itbll with the following command:
> 
> hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList generator 15 
> 100 /tmp/hbase-itbll
> 
> for the generator step and essentially jobs have failed. We can see the meta 
> request are spanning out to replicas, but writes start failing after this due 
> to the stale cache which is not getting updated.
> 
> Would you please tell me more about ‘hfile refresh’ and how to configure it?
> 
> Thanks,
> Andor
> 
> 
> 
> 
> > On 2022. Mar 24., at 17:43, Huaxiang Sun  wrote:
> > 
> > Hi Andor,
> > 
> >   Which 2.4 release do you test in your lab? We use this feature at 
> > production cluster with 2.4.5. 
> > At server side, we use hfile refresh instead of wal replication. I used to 
> > run itbll for each release with this feature enabled. How did you find the 
> > errors, did itbll fail?
> > 
> >   Regards,
> >   Huaxiang
> 
> 

Re: Meta replicas in LoadBalance mode broken in 2.4?

[Huaxiang Sun]

Hi Andor,

   Which 2.4 release do you test in your lab? We use this feature at production 
cluster with 2.4.5. 
At server side, we use hfile refresh instead of wal replication. I used to run 
itbll for each release with this feature enabled. How did you find the errors, 
did itbll fail?

   Regards,
   Huaxiang

[jira] [Created] (HBASE-26864) Region Server does not send Ack back to master after receiving an OpenRegionReq for open regions, causing OpenRegionProcedure stuck forever.

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-26864:


 Summary: Region Server does not send Ack back to master after 
receiving an OpenRegionReq for open regions, causing OpenRegionProcedure stuck 
forever.
 Key: HBASE-26864
 URL: https://issues.apache.org/jira/browse/HBASE-26864
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment
Affects Versions: 2.4.10
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun


For some upgrading cases, we found that master issues RegionOpen for an already 
open region and Region Sever simply logs 
{code:java}
2022-03-17 22:16:55,595 WARN 
org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler: Received OPEN 
for 
foo,b2875fcb-7bc0-4fa9-a980-e902faf7f151,1631771037620.def199cc7208615b783b285f582ddfa4.
 which is already online {code}
and it does not ack or nack master. This OpenRegionProceduce is stuck forever.

In this specific case, it needs to ack master that region is open. 

 

For the cause of why it sent an OpenRegion request for an already open region, 
it will be followed by another issue.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (HBASE-26649) Support meta replica LoadBalance mode for RegionLocator#getAllRegionLocations()

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-26649:


 Summary: Support meta replica LoadBalance mode for 
RegionLocator#getAllRegionLocations()
 Key: HBASE-26649
 URL: https://issues.apache.org/jira/browse/HBASE-26649
 Project: HBase
  Issue Type: Improvement
  Components: meta replicas
Affects Versions: 2.4.9
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun


When HBase application restarts, its meta cache is empty. Normally, it will 
fill the meta cache one region at a time by scanning the meta region. This will 
cause huge pressure to the region server hosting meta during application 
restart. 

It can prefetching all region locations by calling 
RegionLocator#getAllRegionLocations().Meta replica LoadBalance mode is support 
in 2.4, it will be nice to load balance RegionLocator#getAllRegionLocations() 
to all meta replica regions so batch scan can spread across all meta replica 
regions.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Resolved] (HBASE-26590) Hbase-client Meta lookup performance regression between hbase-1 and hbase-2

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-26590.
--
Fix Version/s: 2.5.0
   2.4.10
   Resolution: Fixed

Resolved it for now, will reopen if there is new finding.

> Hbase-client Meta lookup performance regression between hbase-1 and hbase-2
> ---
>
> Key: HBASE-26590
> URL: https://issues.apache.org/jira/browse/HBASE-26590
> Project: HBase
>  Issue Type: Improvement
>  Components: meta
>Affects Versions: 2.4.0, 2.5.0, 2.3.7, 2.6.0
>    Reporter: Huaxiang Sun
>    Assignee: Huaxiang Sun
>Priority: Major
> Fix For: 2.5.0, 2.4.10
>
>
> One of our users complained higher latency after application upgrades from 
> hbase-1.2 client (CDH-5.16.2) to hbase-2.4.5 client with meta replica Load 
> Balance mode during app restart. I reproduced the regression by a test for 
> meta lookup. 
> At my test cluster, there are 160k regions for the test table, so there are 
> 160k entries in meta region. Used one thread to do 1 million meta lookup 
> against the meta region server.
>  
> ||Version ||Meta Replica Load Balance Enabled||Time               ||
> ||2.4.5-with-fixed||Yes||336458ms||
> ||2.4.5-with-fixed||No||333253ms||
> ||2.4.5||Yes||469980ms||
> ||2.4.5||No||470515ms||
> |      *cdh-5.16.2*|                                *No* |  *323412ms*|
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (HBASE-26618) Involving primary meta region in meta scan with Meta Replica Mode

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-26618:


 Summary: Involving primary meta region in meta scan with Meta 
Replica Mode
 Key: HBASE-26618
 URL: https://issues.apache.org/jira/browse/HBASE-26618
 Project: HBase
  Issue Type: Improvement
  Components: meta replicas
Affects Versions: 2.4.9
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun


In the current release with Meta replica LoadBalance mode, the primary meta 
region is not serving the meta scan (only meta replica region serves the read). 
When the result from meta replica region is stale, it will go to primary meta 
region for up-to-date location. 

>From our experience, the primary meta region serves very less read traffic, so 
>it will be better to load balance read traffic across the primary meta region 
>as well.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (HBASE-26590) Hbase-client Meta lookup performance regression between hbase-1 and hbase-2

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-26590:


 Summary: Hbase-client Meta lookup performance regression between 
hbase-1 and hbase-2
 Key: HBASE-26590
 URL: https://issues.apache.org/jira/browse/HBASE-26590
 Project: HBase
  Issue Type: Improvement
  Components: meta
Affects Versions: 2.3.7, 3.0.0-alpha-1
 Environment: ||Version ||Meta Replica Load Balance Enabled||Time       
        ||
||2.4.5-with-fixed||Yes||336458ms||
||2.4.5-with-fixed||No||333253ms||
||2.4.5||Yes||469980ms||
||2.4.5||No||470515ms||
|      *cdh-5.16.2*|                                *No* |  *323412ms*|
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun


One of our users complained higher latency after application upgrades from 
hbase-1.2 client (CDH-5.16.2) to hbase-2.4.5 client with meta replica Load 
Balance mode during app restart. I reproduced the regression by a test for meta 
lookup. 

At my test cluster, there are 160k regions for the test table, so there are 
160k entries in meta region. Used one thread to do 1 million meta lookup 
against the meta region server.

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Resolved] (HBASE-26338) hbck2 setRegionState cannot set replica region state

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-26338.
--
Fix Version/s: hbase-operator-tools-1.2.0
 Release Note: 
To set the replica region's state, it needs the primary region's 
encoded regionname and replica id, the command will be "setRegionState 
, ".
   Resolution: Fixed

> hbck2 setRegionState cannot set replica region state
> 
>
> Key: HBASE-26338
> URL: https://issues.apache.org/jira/browse/HBASE-26338
> Project: HBase
>  Issue Type: Bug
>  Components: hbck2
>Affects Versions: hbase-operator-tools-1.1.0
>Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Major
> Fix For: hbase-operator-tools-1.2.0
>
>
> Currently, there is no way to use hbck2 setRegionState to set a replica 
> region's state, which makes hard to fix inconsistency related with replica 
> regions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-26338) hbck2 setRegionState cannot set replica region state

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-26338:


 Summary: hbck2 setRegionState cannot set replica region state
 Key: HBASE-26338
 URL: https://issues.apache.org/jira/browse/HBASE-26338
 Project: HBase
  Issue Type: Bug
  Components: hbck2
Affects Versions: hbase-operator-tools-1.1.0
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun


Currently, there is no way to use hbck2 setRegionState to set a replica 
region's state, which makes hard to fix inconsistency related with replica 
regions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-26255) Add an option to use region location from meta table in TableSnapshotInputFormat

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-26255.
--
Fix Version/s: 2.4.7
   2.3.7
   3.0.0-alpha-2
   Resolution: Fixed

> Add an option to use region location from meta table in 
> TableSnapshotInputFormat
> 
>
> Key: HBASE-26255
> URL: https://issues.apache.org/jira/browse/HBASE-26255
> Project: HBase
>  Issue Type: Improvement
>  Components: mapreduce
>Affects Versions: 2.3.6
>    Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Major
> Fix For: 3.0.0-alpha-2, 2.3.7, 2.4.7
>
>
> TableSnapshotInputFormat currently calculates block locality of a region to 
> decide the best location to run the task. While this works for a small scale 
> table snapshot, we found that for a table snapshot with many regions, the 
> locality calculation takes too much time. 
> In the case of a table with high locality, we can use region location from 
> meta table to decide a snapshot region's location. Add an option to support 
> it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-26272) TestTableMapReduceUntil failure in branch-2

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-26272:


 Summary: TestTableMapReduceUntil failure in branch-2
 Key: HBASE-26272
 URL: https://issues.apache.org/jira/browse/HBASE-26272
 Project: HBase
  Issue Type: Test
  Components: test
Reporter: Huaxiang Sun


 
{code:java}
[ERROR] 
org.apache.hadoop.hbase.mapreduce.TestTableMapReduceUtil.testInitCredentialsForCluster3
  Time elapsed: 8.122 s  <<< ERROR!
org.apache.hadoop.security.KerberosAuthException: Login failure for user: 
hsun/localh...@example.com from keytab 
/Users/hsun/work/hbase-hs/hbase-1/hbase-mapreduce/target/test-data/b12f4926-d8ec-1129-0101-1ba76e65f3c2/keytab
 javax.security.auth.login.LoginException: java.lang.IllegalArgumentException: 
Illegal principal name hsun/localh...@example.com: 
org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No 
rules applied to hsun/localh...@example.com
 at 
org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1104)
 at 
org.apache.hadoop.hbase.mapreduce.TestTableMapReduceUtil.testInitCredentialsForCluster3(TestTableMapReduceUtil.java:233)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
 at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
 at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
 at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
 at 
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
 at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
 at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
 at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
 at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
 at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
 at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
 at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
 at org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:38)
 at 
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:288)
 at 
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:282)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.lang.Thread.run(Thread.java:748)
Caused by: javax.security.auth.login.LoginException: 
java.lang.IllegalArgumentException: Illegal principal name 
hsun/localh...@example.com: 
org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No 
rules applied to hsun/localh...@example.com
 at 
org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.commit(UserGroupInformation.java:224)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
 at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
 at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
 at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
 at javax.security.auth.login.LoginContext.login(LoginContext.java:588)
 at 
org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1095)
 ... 25 more
Caused by: java.lang.IllegalArgumentException: Illegal principal name 
hsun/localh...@example.com: 
org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No 
rules applied to hsun/localh...@example.com
 at org.apache.hadoop.security.User.(User.java:50)
 at org.apache.hadoop.security.User.(User.java:43)
 at 
org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.commit(UserGroupInformation.java:222)
 ... 37 more
Caused by: 
org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No 
rules applied to hsun/localh...@ex

[jira] [Created] (HBASE-26255) Add an option to use region location from meta table in TableSnapshotInputFormat

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-26255:


 Summary: Add an option to use region location from meta table in 
TableSnapshotInputFormat
 Key: HBASE-26255
 URL: https://issues.apache.org/jira/browse/HBASE-26255
 Project: HBase
  Issue Type: Improvement
  Components: mapreduce
Affects Versions: 2.3.6
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun


TableSnapshotInputFormat currently calculates block locality of a region to 
decide the best location to run the task. While this works for a small scale 
table snapshot, we found that for a table snapshot with many regions, the 
locality calculation takes too much time. 

In the case of a table with high locality, we can use region location from meta 
table to decide a snapshot region's location. Add an option to support it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] Second release candidate for HBASE 2.3.6 (RC3)

[Huaxiang Sun]

+1

~/work/hbase

* Signature: ok

* Checksum : ok

* Rat check (1.8.0_242): ok

 - mvn clean apache-rat:check

* Built from source (1.8.0_242): ok

 - mvn clean install  -DskipTests

* Unit tests pass (1.8.0_242): ok

 - mvn package -P runSmallTests  -Dsurefire.rerunFailingTestsCount=3


Something messed up in my local environment. I cleaned up all local
directories and rerun the vote script. It passed.
I also checked CHANGES.md, it is updated and looks good.




On Thu, Jul 29, 2021 at 9:29 AM clara xiong  wrote:

> +1
>
> * Signature: ok
>
> * Checksum : ok
>
> * Rat check (1.8.0_261): ok
>
>  - mvn clean apache-rat:check -D skipTests
>
> * Built from source (1.8.0_261): ok
>
>  - mvn clean install -D skipTests -DskipTests
>
> * Unit tests pass (1.8.0_261): ok
>
>  - mvn package -P runAllTests -D skipTests
> -Dsurefire.rerunFailingTestsCount=3
>
>
>
> On Wed, Jul 28, 2021 at 10:58 PM Huaxiang Sun 
> wrote:
>
> > I got a checksum error, is it just me? Otherwise, I checked release file
> > and jiras there look good.
> >
> > ~/work/hbase
> >
> > * Signature: ok
> >
> > * Checksum : failed
> >
> > * Rat check (1.8.0_242): ok
> >
> >  - mvn clean apache-rat:check
> >
> > * Built from source (1.8.0_242): ok
> >
> >  - mvn clean install  -DskipTests
> >
> > * Unit tests pass (1.8.0_242): ok
> >
> >  - mvn package -P runSmallTests
> > -Dsurefire.rerunFailingTestsCount=3
> >
> > iMac-Pro:hbase hsun$ ./dev-support/hbase-vote.sh -s
> > https://dist.apache.org/repos/dist/dev/hbase/2.3.6RC3 -P runSmallTests
> >
> > On Wed, Jul 28, 2021 at 9:20 PM Stack  wrote:
> >
> > > On Wed, Jul 28, 2021 at 9:16 PM Stack  wrote:
> > >
> > > > Looks like. Let me fix.
> > > > S
> > > >
> > > > Done.
> > > Thanks,
> > > S
> > >
> > >
> > >
> > > > On Wed, Jul 28, 2021 at 8:27 PM 张铎(Duo Zhang)  >
> > > > wrote:
> > > >
> > > >> Not related to vote but I saw a 2.3.6 branch in the git repo?
> Mistake?
> > > >>
> > > >> Stack  于2021年7月29日周四 上午10:56写道：
> > > >> >
> > > >> > Please vote on this Apache hbase release candidate,
> > > >> > hbase-2.3.6RC3
> > > >> >
> > > >> > The VOTE will remain open for at least 72 hours.
> > > >> >
> > > >> > [ ] +1 Release this package as Apache hbase 2.3.6
> > > >> > [ ] -1 Do not release this package because ...
> > > >> >
> > > >> > The tag to be voted on is 2.3.6RC3:
> > > >> >
> > > >> >   https://github.com/apache/hbase/tree/2.3.6RC3
> > > >> >
> > > >> > This tag currently points to git reference
> > > >> >
> > > >> >   7414579f2620fca6b75146c29ab2726fc4643ac9
> > > >> >
> > > >> > The release files, including signatures, digests, as well as
> > > CHANGES.md
> > > >> > and RELEASENOTES.md included in this RC can be found at:
> > > >> >
> > > >> >   https://dist.apache.org/repos/dist/dev/hbase/2.3.6RC3/
> > > >> >
> > > >> > Maven artifacts are available in a staging repository at:
> > > >> >
> > > >> >
> > > >>
> > https://repository.apache.org/content/repositories/orgapachehbase-1463/
> > > >> >
> > > >> > Artifacts were signed with the 0xA7874F29 key which can be found
> in:
> > > >> >
> > > >> >   https://dist.apache.org/repos/dist/release/hbase/KEYS
> > > >> >
> > > >> > To learn more about Apache hbase, please see
> > > >> >
> > > >> >   http://hbase.apache.org/
> > > >> >
> > > >> > Thanks,
> > > >> > Your HBase Release Manager
> > > >>
> > > >
> > >
> >
>

Re: [VOTE] Second release candidate for HBase 2.4.5 (RC1) is available

[Huaxiang Sun]

+1

* Signature: ok

* Checksum : ok

* Rat check (1.8.0_242): ok

 - mvn clean apache-rat:check

* Built from source (1.8.0_242): ok

 - mvn clean install  -DskipTests

* Unit tests pass (1.8.0_242): ok

 - mvn package -P runSmallTests  -Dsurefire.rerunFailingTestsCount=3

iMac-Pro:hbase hsun$

On Tue, Jul 27, 2021 at 4:39 PM Stack  wrote:

> +1
>
> * Signature: ok
> * Checksum : ok
> * Rat check (1.8.0_292): ok
>  - mvn clean apache-rat:check
> * Built from source (1.8.0_292): ok
>  - mvn clean install  -DskipTests
> * Unit tests pass (1.8.0_292): ok
>  - mvn package -P runSmallTests
> -Dsurefire.rerunFailingTestsCount=3
>
> Compat report looks good.
>
> Built from src w/ release profile. Started it up in standalone mode. Loaded
> up data w/ pe. Verified it all made it. Poked around UI and shell. Looks
> good to me.
>
> S
>
> On Tue, Jul 27, 2021 at 11:09 AM Andrew Purtell 
> wrote:
>
> > Please vote on this Apache hbase release candidate,
> > hbase-2.4.5RC1
> >
> > The VOTE will remain open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache hbase 2.4.5
> > [ ] -1 Do not release this package because ...
> >
> > The tag to be voted on is 2.4.5RC1:
> >
> >   https://github.com/apache/hbase/tree/2.4.5RC1
> >
> > This tag currently points to git reference 03b8c0cf42.
> >
> > The release files, including signatures, digests, as well as CHANGES.md
> > and RELEASENOTES.md included in this RC can be found at:
> >
> >   https://dist.apache.org/repos/dist/dev/hbase/2.4.5RC1/
> >
> > The compatibility report for this RC can be found at:
> >
> >
> >
> https://dist.apache.org/repos/dist/dev/hbase/2.4.5RC1/api_compare_2.4.4_to_2.4.5RC1.html
> >
> > There are 0 reported compatibility issues.
> >
> > Maven artifacts are available in a staging repository at:
> >
> >
> https://repository.apache.org/content/repositories/orgapachehbase-1462/
> >
> > Artifacts were signed with the 0xD5365CCD key which can be found in:
> >
> >   https://dist.apache.org/repos/dist/release/hbase/KEYS
> >
> > To learn more about Apache hbase, please see
> >
> >   http://hbase.apache.org/
> >
> > Thanks,
> > Your HBase Release Manager
> >
>

Re: [VOTE] Second release candidate for HBASE 2.3.6 (RC3)

[Huaxiang Sun]

I got a checksum error, is it just me? Otherwise, I checked release file
and jiras there look good.

~/work/hbase

* Signature: ok

* Checksum : failed

* Rat check (1.8.0_242): ok

 - mvn clean apache-rat:check

* Built from source (1.8.0_242): ok

 - mvn clean install  -DskipTests

* Unit tests pass (1.8.0_242): ok

 - mvn package -P runSmallTests  -Dsurefire.rerunFailingTestsCount=3

iMac-Pro:hbase hsun$ ./dev-support/hbase-vote.sh -s
https://dist.apache.org/repos/dist/dev/hbase/2.3.6RC3 -P runSmallTests

On Wed, Jul 28, 2021 at 9:20 PM Stack  wrote:

> On Wed, Jul 28, 2021 at 9:16 PM Stack  wrote:
>
> > Looks like. Let me fix.
> > S
> >
> > Done.
> Thanks,
> S
>
>
>
> > On Wed, Jul 28, 2021 at 8:27 PM 张铎(Duo Zhang) 
> > wrote:
> >
> >> Not related to vote but I saw a 2.3.6 branch in the git repo? Mistake?
> >>
> >> Stack  于2021年7月29日周四 上午10:56写道：
> >> >
> >> > Please vote on this Apache hbase release candidate,
> >> > hbase-2.3.6RC3
> >> >
> >> > The VOTE will remain open for at least 72 hours.
> >> >
> >> > [ ] +1 Release this package as Apache hbase 2.3.6
> >> > [ ] -1 Do not release this package because ...
> >> >
> >> > The tag to be voted on is 2.3.6RC3:
> >> >
> >> >   https://github.com/apache/hbase/tree/2.3.6RC3
> >> >
> >> > This tag currently points to git reference
> >> >
> >> >   7414579f2620fca6b75146c29ab2726fc4643ac9
> >> >
> >> > The release files, including signatures, digests, as well as
> CHANGES.md
> >> > and RELEASENOTES.md included in this RC can be found at:
> >> >
> >> >   https://dist.apache.org/repos/dist/dev/hbase/2.3.6RC3/
> >> >
> >> > Maven artifacts are available in a staging repository at:
> >> >
> >> >
> >> https://repository.apache.org/content/repositories/orgapachehbase-1463/
> >> >
> >> > Artifacts were signed with the 0xA7874F29 key which can be found in:
> >> >
> >> >   https://dist.apache.org/repos/dist/release/hbase/KEYS
> >> >
> >> > To learn more about Apache hbase, please see
> >> >
> >> >   http://hbase.apache.org/
> >> >
> >> > Thanks,
> >> > Your HBase Release Manager
> >>
> >
>

Re: [VOTE] First release candidate for HBase 2.3.6 (RC2) is available

[Huaxiang Sun]

Sorry, I have to put -1 on this RC.

In release note,
https://dist.apache.org/repos/dist/dev/hbase/2.3.6RC2/CHANGES.md,
HBASE-24492 is listed in 2.3.6.
Actually, this jira is in 2.3.3, not 2.3.6. The release version in the jira
is messed up. I corrected it in the jira.

| [HBASE-24492](https://issues.apache.org/jira/browse/HBASE-24492) |
ProtobufLogReader.readNext does not need looping |  Minor |
Replication, wal |

Huaxiang


On Tue, Jul 27, 2021 at 4:06 PM Nick Dimiduk  wrote:

> +1
>
> * Compatibility report: ok
> * Nightly builds good: okay
> * Signature: ok
> * Checksum : ok
> * Rat check (1.8.0_292): ok
>  - mvn clean apache-rat:check -D skipTests
> * Rat check (11.0.11): ok
>  - mvn clean apache-rat:check -D skipTests -D hadoop.profile=3.0
> * Built from source (1.8.0_292): ok
>  - mvn clean install -D skipTests -DskipTests
> * Built from source (11.0.11): ok
>  - mvn clean install -D skipTests -D hadoop.profile=3.0 -DskipTests
> * Unit tests pass (1.8.0_292): ok
>  - mvn package -P runAllTests -D skipTests
> -Dsurefire.rerunFailingTestsCount=3
> * Unit tests pass (11.0.11): ok
>  - mvn package -P runAllTests -D skipTests -D hadoop.profile=3.0
> -Dsurefire.rerunFailingTestsCount=3
>
> On Mon, Jul 26, 2021 at 11:14 PM Stack  wrote:
>
> > Please vote on this Apache hbase release candidate,
> > hbase-2.3.6RC2
> >
> > The VOTE will remain open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache hbase 2.3.6
> > [ ] -1 Do not release this package because ...
> >
> > The tag to be voted on is 2.3.6RC2:
> >
> >   https://github.com/apache/hbase/tree/2.3.6RC2
> >
> > This tag currently points to git reference
> >
> >   79d57a453ac840873ca15fc151b6f9f342b467b9
> >
> > The release files, including signatures, digests, as well as CHANGES.md
> > and RELEASENOTES.md included in this RC can be found at:
> >
> >   https://dist.apache.org/repos/dist/dev/hbase/2.3.6RC2/
> >
> > Maven artifacts are available in a staging repository at:
> >
> >
> https://repository.apache.org/content/repositories/orgapachehbase-1461/
> >
> > Artifacts were signed with the 0xA7874F29 key which can be found in:
> >
> >   https://dist.apache.org/repos/dist/release/hbase/KEYS
> >
> > Known issues to be aware of:
> >
> >  HBASE-26120 New replication gets stuck or data loss when multiwal groups
> > more than 10
> >
> > To learn more about Apache hbase, please see
> >
> >   http://hbase.apache.org/
> >
> > Thanks,
> > Your HBase Release Manager
> >
>

[jira] [Resolved] (HBASE-26108) add option to disable scanMetrics in TableSnapshotInputFormat

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-26108.
--
Fix Version/s: 2.4.5
   3.0.0-alpha-2
   2.3.6
   Resolution: Fixed

> add option to disable scanMetrics in TableSnapshotInputFormat
> -
>
> Key: HBASE-26108
> URL: https://issues.apache.org/jira/browse/HBASE-26108
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.3.5
>        Reporter: Huaxiang Sun
>    Assignee: Huaxiang Sun
>Priority: Major
> Fix For: 2.3.6, 3.0.0-alpha-2, 2.4.5
>
>
> When running spark job with TableSnapshotInputFormat, we found that scan is 
> very slower. We found that scanMetrics is hardcoded as enabled, spark's 
> newAPIHadoopRDD uses DummyReporter in hadoop, which causes the following 
> exception and 80% cpu time is spent on this exception handling. 
> Need to provide an option to disable scanMetrics.
> java.base@11.0.5/java.lang.Throwable.fillInStackTrace(Native Method)
> java.base@11.0.5/java.lang.Throwable.fillInStackTrace(Throwable.java:787) => 
> holding Monitor(java.util.MissingResourceException@258206255})
> java.base@11.0.5/java.lang.Throwable.(Throwable.java:292)
> java.base@11.0.5/java.lang.Exception.(Exception.java:84)
> java.base@11.0.5/java.lang.RuntimeException.(RuntimeException.java:80)
> java.base@11.0.5/java.util.MissingResourceException.(MissingResourceException.java:85)
> java.base@11.0.5/java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:2055)
> java.base@11.0.5/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1689)
> java.base@11.0.5/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1593)
> java.base@11.0.5/java.util.ResourceBundle.getBundle(ResourceBundle.java:1284)
> app//org.apache.hadoop.mapreduce.util.ResourceBundles.getBundle(ResourceBundles.java:37)
> app//org.apache.hadoop.mapreduce.util.ResourceBundles.getValue(ResourceBundles.java:56)
>  => holding Monitor(java.lang.Class@545605549})
> app//org.apache.hadoop.mapreduce.util.ResourceBundles.getCounterGroupName(ResourceBundles.java:77)
> app//org.apache.hadoop.mapreduce.counters.CounterGroupFactory.newGroup(CounterGroupFactory.java:94)
> app//org.apache.hadoop.mapreduce.counters.AbstractCounters.getGroup(AbstractCounters.java:227)
> app//org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:154)
> app//org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl$DummyReporter.getCounter(TaskAttemptContextImpl.java:110)
> app//org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl.getCounter(TaskAttemptContextImpl.java:76)
> org.apache.hadoop.hbase.mapreduce.TableRecordReaderImpl.updateCounters(TableRecordReaderImpl.java:311)
> org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormat$TableSnapshotRegionRecordReader.nextKeyValue(TableSnapshotInputFormat.java:167)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-26108) add option to disable scanMetrics in TableSnapshotInputFormat

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-26108:


 Summary: add option to disable scanMetrics in 
TableSnapshotInputFormat
 Key: HBASE-26108
 URL: https://issues.apache.org/jira/browse/HBASE-26108
 Project: HBase
  Issue Type: Improvement
Affects Versions: 2.3.5
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun


When running spark job with TableSnapshotInputFormat, we found that scan is 
very slower. We found that scanMetrics is hardcoded as enabled, spark's 

newAPIHadoopRDD uses DummyReporter in hadoop, which causes the following 
exception and 80% cpu time is spent on this exception handling. 

Need to provide an option to disable scanMetrics.
java.base@11.0.5/java.lang.Throwable.fillInStackTrace(Native Method)
java.base@11.0.5/java.lang.Throwable.fillInStackTrace(Throwable.java:787) => 
holding Monitor(java.util.MissingResourceException@258206255})
java.base@11.0.5/java.lang.Throwable.(Throwable.java:292)
java.base@11.0.5/java.lang.Exception.(Exception.java:84)
java.base@11.0.5/java.lang.RuntimeException.(RuntimeException.java:80)
java.base@11.0.5/java.util.MissingResourceException.(MissingResourceException.java:85)
java.base@11.0.5/java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:2055)
java.base@11.0.5/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1689)
java.base@11.0.5/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1593)
java.base@11.0.5/java.util.ResourceBundle.getBundle(ResourceBundle.java:1284)
app//org.apache.hadoop.mapreduce.util.ResourceBundles.getBundle(ResourceBundles.java:37)
app//org.apache.hadoop.mapreduce.util.ResourceBundles.getValue(ResourceBundles.java:56)
 => holding Monitor(java.lang.Class@545605549})
app//org.apache.hadoop.mapreduce.util.ResourceBundles.getCounterGroupName(ResourceBundles.java:77)
app//org.apache.hadoop.mapreduce.counters.CounterGroupFactory.newGroup(CounterGroupFactory.java:94)
app//org.apache.hadoop.mapreduce.counters.AbstractCounters.getGroup(AbstractCounters.java:227)
app//org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:154)
app//org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl$DummyReporter.getCounter(TaskAttemptContextImpl.java:110)
app//org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl.getCounter(TaskAttemptContextImpl.java:76)
org.apache.hadoop.hbase.mapreduce.TableRecordReaderImpl.updateCounters(TableRecordReaderImpl.java:311)
org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormat$TableSnapshotRegionRecordReader.nextKeyValue(TableSnapshotInputFormat.java:167)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [ANNOUNCE] New HBase committer Baiqiang Zhao

[Huaxiang Sun]

Congratulations Baiqiang!

On Tue, Jul 13, 2021 at 11:10 AM Zach York 
wrote:

> Congratulations and welcome Baiqiang!
>
> On Mon, Jul 12, 2021 at 4:59 AM Baiqiang Zhao  wrote:
>
> > Thank you all for your warm welcome!
> >
> > Peter Somogyi  于2021年7月12日周一 下午7:46写道：
> >
> > > Congratulations!
> > >
> > > On Mon, Jul 12, 2021 at 12:53 PM Yu Li  wrote:
> > >
> > > > Congrats and welcome, Baiqiang.
> > > >
> > > > Best Regards,
> > > > Yu
> > > >
> > > >
> > > > On Mon, 12 Jul 2021 at 16:56, 哈晓琳  wrote:
> > > >
> > > > > Congratulations Baiqiang!
> > > > >
> > > > > Balazs Meszaros 
> 于2021年7月12日周一
> > > > > 下午4:35写道：
> > > > >
> > > > > > Congratulations Baiqiang!
> > > > > >
> > > > > > On Mon, Jul 12, 2021 at 9:22 AM Pankaj Kumar <
> > pankajku...@apache.org
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Congratulations Baiqiang.!!
> > > > > > >
> > > > > > > Regards,
> > > > > > > Pankaj
> > > > > > >
> > > > > > > On Sun, Jul 11, 2021, 12:48 AM Nick Dimiduk <
> ndimi...@apache.org
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Hi everyone,
> > > > > > > >
> > > > > > > > On behalf of the Apache HBase PMC I am pleased to announce
> that
> > > > > > Baiqiang
> > > > > > > > Zhao has accepted the PMC's invitation to become a committer
> on
> > > the
> > > > > > > > project!
> > > > > > > >
> > > > > > > > We appreciate all of the great contributions Baiqiang has
> made
> > to
> > > > > > > > the community thus far and we look forward to his continued
> > > > > > involvement.
> > > > > > > >
> > > > > > > > Allow me to be the first to congratulate Baiqiang on his new
> > > role!
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Nick
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Time for 2.4.5

[Huaxiang Sun]

Sorry, typo. s/we run into this core dump quite often as wekk/we run into
this core dump quite often as well.

On Fri, Jul 16, 2021 at 10:58 AM Huaxiang Sun  wrote:

> Hi Andrew,
>
>Do we want to wait a few days for HBASE-24984
> <https://issues.apache.org/jira/browse/HBASE-24984>? Several users (we
> run into this core dump quite often as wekk) reported core dump and a fix
> is ready.
>
>Thanks,
>Huaxiang
>
> On Sat, Jul 10, 2021 at 7:35 PM Andrew Purtell 
> wrote:
>
>> It’s about time for 2.4.5.
>>
>> If there is pending work you’d like to see go out in the release, please
>> commit it in the next couple of days. Any potential blockers, please raise
>> them here in this thread.
>>
>> Planning to have an RC by the end of the week.
>>
>>

Re: Time for 2.4.5

[Huaxiang Sun]

Hi Andrew,

   Do we want to wait a few days for HBASE-24984
? Several users (we run
into this core dump quite often as wekk) reported core dump and a fix is
ready.

   Thanks,
   Huaxiang

On Sat, Jul 10, 2021 at 7:35 PM Andrew Purtell 
wrote:

> It’s about time for 2.4.5.
>
> If there is pending work you’d like to see go out in the release, please
> commit it in the next couple of days. Any potential blockers, please raise
> them here in this thread.
>
> Planning to have an RC by the end of the week.
>
>

[jira] [Created] (HBASE-26092) JVM core dump in the replication path

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-26092:


 Summary: JVM core dump in the replication path
 Key: HBASE-26092
 URL: https://issues.apache.org/jira/browse/HBASE-26092
 Project: HBase
  Issue Type: Bug
  Components: Replication
Affects Versions: 2.3.5
Reporter: Huaxiang Sun


When replication is turned on, we found the following code dump in the region 
server. 

I checked the code dump for replication. I think I got some ideas. For 
replication, when RS receives walEdits from remote cluster, it needs to send 
them out to final RS. In this case, NettyRpcConnection is deployed, calls are 
queued while it refers to ByteBuffer in the context of replicationHandler 
(returned to the pool once it returns). Code dump will happen since the 
byteBuffer has been reused. Needs ref count in this asynchronous processing.

 

Feel free to take it, otherwise, I will try to work on a patch later.

 

 
{code:java}
Stack: [0x7fb1bf039000,0x7fb1bf13a000],  sp=0x7fb1bf138560,  free 
space=1021k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
J 28175 C2 
org.apache.hadoop.hbase.ByteBufferKeyValue.write(Ljava/io/OutputStream;Z)I (21 
bytes) @ 0x7fd2663c [0x7fd263c0+0x27c]
J 14912 C2 
org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.writeRequest(Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelHandlerContext;Lorg/apache/hadoop/hbase/ipc/Call;Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelPromise;)V
 (370 bytes) @ 0x7fdbbb94b590 [0x7fdbbb949c00+0x1990]
J 14911 C2 
org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.write(Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelHandlerContext;Ljava/lang/Object;Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelPromise;)V
 (30 bytes) @ 0x7fdbb972d1d4 [0x7fdbb972d1a0+0x34]
J 30476 C2 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.write(Ljava/lang/Object;ZLorg/apache/hbase/thirdparty/io/netty/channel/ChannelPromise;)V
 (149 bytes) @ 0x7fdbbd4e7084 [0x7fdbbd4e6900+0x784]
J 14914 C2 org.apache.hadoop.hbase.ipc.NettyRpcConnection$6$1.run()V (22 bytes) 
@ 0x7fdbbb9344ec [0x7fdbbb934280+0x26c]
J 23528 C2 
org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(J)Z
 (106 bytes) @ 0x7fdbbcbb0efc [0x7fdbbcbb0c40+0x2bc]
J 15987% C2 
org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run()V (461 
bytes) @ 0x7fdbbbaf1580 [0x7fdbbbaf1360+0x220]
j  
org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run()V+44
j  
org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run()V+11
j  
org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run()V+4
{code}
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [ANNOUNCE] New HBase Committer Xiaolin Ha(哈晓琳)

[Huaxiang Sun]

Congratulations, Xiaolin Ha!

On Sun, May 16, 2021 at 7:33 AM 哈晓琳  wrote:

> It's my pleasure to participate in this project, and I'll keep
> contributing, thanks.
>
> Yulin Niu  于2021年5月16日周日 上午10:06写道：
>
> > Congratulations, Xiaolin Ha!
> >
> > Andrew Purtell  于2021年5月16日周日 上午4:38写道：
> >
> > > Welcome, Xiaolin Ha.
> > >
> > > On Sat, May 15, 2021 at 7:11 AM 张铎(Duo Zhang) 
> > > wrote:
> > >
> > > > On behalf of the Apache HBase PMC, I am pleased to announce that
> > Xiaolin
> > > > Ha(sunhelly) has accepted the PMC's invitation to become a committer
> on
> > > the
> > > > project. We appreciate all of Xiaolin's generous contributions thus
> far
> > > and
> > > > look forward to her continued involvement.
> > > >
> > > > Congratulations and welcome, Xiaolin Ha!
> > > >
> > > > 我很高兴代表Apache HBase PMC宣布哈晓琳已接受我们的邀请，成为Apache
> > > > HBase项目的Committer。感谢哈晓琳一直以来为HBase项目做出的贡献，并期待她在未来继续承担更多的责任。
> > > >
> > > > 欢迎哈晓琳！
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew
> > >
> > > Words like orphans lost among the crosstalk, meaning torn from truth's
> > > decrepit hands
> > >- A23, Crosstalk
> > >
> >
>

Re: [DISCUSS] User experience on Stripe Compaction

[Huaxiang Sun]

It is great to hear successful stories from large scale production
deployment!

On Mon, May 3, 2021 at 1:31 PM clara xiong  wrote:

> It is great to hear this is still in use  in a large production and getting
> improvement!
>
> On Tue, Apr 27, 2021 at 3:01 AM 张铎(Duo Zhang) 
> wrote:
>
> > Stripe compaction is a feature which has been implemented long ago in
> HBase
> > but I've never seen extensive usage in the community. And recently I
> found
> > that a big company in China, Meituan, has made use of stripe compaction
> in
> > their production cluster. One of the team member shared some information
> on
> > a github PR and she agreed that I could share it in the mailing list.
> >
> > Hi, @Apache9  , it's my pleasure to share
> > these
> > > information.
> > > We use StripeCompactionPolicy in almost all of our production clusters.
> > > And we let recently data in memstore flush to L0, limit stripe size to
> > > about 10G. Most of our regions are large than 50G, there even exist
> > regions
> > > larger than 2T... StripeCompactionPolicy has no major compactions, and
> it
> > > can limit compactions in only L0 and one stripe files, and it can
> perform
> > > cells deletion in one stripe just like the major-compaction. The
> pressure
> > > of compactions is broken down. Though the total files count in a region
> > > maybe a little larger than normal compactions, because the files are
> > > organized as in mini-regions, it works well for most read requests.
> > > And we also implemented a fast split and compact method based on
> > > StripeCompactionPolicy, practiced in all our production clusters,
> results
> > > show that split is very light weight and no need to perform read+write
> > > files compactions right after split. Details are in HBASE-25302
> > > , hope you have
> > > interest...
> > > Thanks.
> >
> >
> > The original link:
> > https://github.com/apache/hbase/pull/3152#issuecomment-824166990
> >
>

Re: [ANNOUNCE] New HBase PMC Huaxiang Sun

[Huaxiang Sun]

Thanks all!

Huaxiang



On Tue, Apr 13, 2021 at 9:12 AM Esteban Gutierrez
 wrote:

> Congratulations Huaxiang!
>
>
> --
> Cloudera, Inc.
>
>
>
> On Tue, Apr 13, 2021 at 8:16 AM zheng wang <18031...@qq.com> wrote:
>
> > Congratulations!
> >
> >
> >
> >
> > --原始邮件--
> > 发件人:
> >   "user"
> > <
> > vjas...@apache.org;
> > 发送时间:2021年4月13日(星期二) 下午4:09
> > 收件人:"hbase-dev" > u...@hbase.apache.org;
> >
> > 主题:[ANNOUNCE] New HBase PMC Huaxiang Sun
> >
> >
> >
> > On behalf of the Apache HBase PMC I am pleased to announce that Huaxiang
> > Sun has accepted our invitation to become a PMC member on the HBase
> > project. We appreciate Huaxiang stepping up to take more responsibility
> for
> > the project.
> >
> > Congratulations and welcome, Huaxiang!
>

Re: EOL branch-1 and all 1.x ?

[Huaxiang Sun]

> do you have a JIRA covering this change? What is your suggested
> remediation for branch-2 releases? Can/should we make the fix the default
> behavior, or is there good reason to keep us running in this “slow mode”
by
> default, require operators to opt-in to the perf boost?

The change of Rpc PriorityQueue scheduler
from FastPathBalancedQueueRpcExecutor to RWQueueRpcExecutor causes
meta scan throughput regression. This change is introduced by HBASE-22280,
I am about to follow up in HBASE-22280.
Could be a real production issue addressed by this change, not sure at this
moment.


On Thu, Apr 1, 2021 at 11:11 AM Nick Dimiduk  wrote:

> Also big +1 from me.
>
> On Thu, Apr 1, 2021 at 09:08 Huaxiang Sun  wrote:
>
> > Per performance regression concern, we had one such issue for meta when
> > upgrading from 1.2 to 2.3.
> > It turned out to be default rpc scheduling changed from branch-1 to
> > branch-2, and it causes performance regression.
>
>
> Huaxiang, do you have a JIRA covering this change? What is your suggested
> remediation for branch-2 releases? Can/should we make the fix the default
> behavior, or is there good reason to keep us running in this “slow mode” by
> default, require operators to opt-in to the perf boost?
>
> Thanks,
> Nick
>
> On Thu, Apr 1, 2021 at 8:22 AM Pankaj Kumar 
> > wrote:
> >
> > > +1 on EOL branch-1 and all branch-1.x.
> > >
> > > Regards,
> > > Pankaj
> > >
> > > On Thu, Apr 1, 2021 at 3:34 AM Andrew Purtell 
> > wrote:
> > >
> > > > Is it time to consider EOL of branch-1 and all 1.x releases ?
> > > >
> > > > There doesn't seem to be much developer interest in branch-1 beyond
> > > > occasional maintenance. This is understandable. Per our compatibility
> > > > guidelines, branch-1 commits must be compatible with Java 7, and the
> > > range
> > > > of acceptable versions of third party dependencies is also restricted
> > due
> > > > to Java 7 compatibility requirements. Most developers are writing
> code
> > > with
> > > > Java 8+ idioms these days. For that reason and because the branch-1
> > code
> > > > base is generally aged at this point, all but trivial (or lucky!)
> > > backports
> > > > require substantial changes in order to integrate adequately. Let me
> > also
> > > > observe that branch-1 artifacts are not fully compatible with Java 11
> > or
> > > > later. (The shell is a good example of such issues: The version of
> > > > jruby-complete required by branch-1 is not compatible with Java 11
> and
> > > > upgrading to the version used by branch-2 causes shell commands to
> > error
> > > > out due to Ruby language changes.)
> > > >
> > > > We can a priori determine there is insufficient motivation for
> > production
> > > > of release artifacts for the PMC to vote upon. Otherwise, someone
> would
> > > > have done it. We had 12 releases from branch-2 derived code in 2019,
> 13
> > > > releases from branch-2 derived code in 2020, and so far we have had 3
> > > > releases from branch-2 derived code in 2021. In contrast, we had 8
> > > releases
> > > > from branch-1 derived code in 2019, 0 releases from branch-1 in 2020,
> > and
> > > > so far 0 releases from branch-1 in 2021.
> > > >
> > > > *  2021202020191.x0282.x31312*
> > > >
> > > > If there is someone interested in continuing branch-1, now is the
> time
> > to
> > > > commit. However let me be clear that simply expressing an abstract
> > desire
> > > > to see continued branch-1 releases will not be that useful. It will
> be
> > > > noted, but will not have much real world impact. Apache is a
> do-ocracy.
> > > In
> > > > the absence of intrinsic motivation of project participants, which is
> > > what
> > > > we seem to have here, you will need to do something: Fix the
> > > compatibility
> > > > issues, if any between the last release of 1.x and the current
> branch-1
> > > > head; fix any failing and flaky unit tests; produce release
> artifacts;
> > > and
> > > > submit those artifacts to the PMC for voting. Or, convince someone
> with
> > > > commit rights and/or PMC membership to undertake these actions on
> your
> > > > behalf.
> > > >
> > > > Otherwise, I respectfully submit for your consideration, it is time
> to
> > > > declare  branch-1 and all 1.x code lines EOL, simply acknowledging
> what
> > > has
> > > > effectively already happened.
> > > >
> > > > --
> > > > Best regards,
> > > > Andrew
> > > >
> > > > Words like orphans lost among the crosstalk, meaning torn from
> truth's
> > > > decrepit hands
> > > >- A23, Crosstalk
> > > >
> > >
> >
>

[ANNOUNCE] Apache HBase 2.3.5 is now available for download

[Huaxiang Sun]

The HBase team is happy to announce the immediate availability of HBase

2.3.5.


Apache HBase™ is an open-source, distributed, versioned, non-relational

database.

Apache HBase gives you low latency random access to billions of rows with

millions of columns atop non-specialized hardware. To learn more about

HBase, see https://hbase.apache.org/.


HBase 2.3.5 is the fifth patch release in the HBase 2.3.x line, which aims

to improve the stability and reliability of HBase. This release includes 53
bug

fixes and improvements since 2.3.4.


The full list of issues and release notes can be found here:

CHANGELOG: https://downloads.apache.org/hbase/2.3.5/CHANGES.md

RELEASENOTES: https://downloads.apache.org/hbase/2.3.5/RELEASENOTES.md


or via our issue tracker:

https://issues.apache.org/jira/projects/HBASE/versions/12349549


To download please follow the links and instructions on our website:


https://hbase.apache.org/downloads.html


Questions, comments, and problems are always welcome at:

dev@hbase.apache.org

u...@hbase.apache.org


Thanks to all who contributed and made this release possible.


Cheers,

The HBase Dev Team

[jira] [Created] (HBASE-25724) update download area for 2.3.5 as new stable build

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-25724:


 Summary: update download area for 2.3.5 as new stable build
 Key: HBASE-25724
 URL: https://issues.apache.org/jira/browse/HBASE-25724
 Project: HBase
  Issue Type: Sub-task
  Components: community
Reporter: Sean Busbey
Assignee: Sean Busbey



* update the stable symlink to point to 2.3.4
* Remove the 2.3.3 release from downloads.a.o



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: EOL branch-1 and all 1.x ?

[Huaxiang Sun]

+1 on EOL branch-1 and all branch-1.x.

Per performance regression concern, we had one such issue for meta when
upgrading from 1.2 to 2.3.
It turned out to be default rpc scheduling changed from branch-1 to
branch-2, and it causes performance regression.



On Thu, Apr 1, 2021 at 8:22 AM Pankaj Kumar 
wrote:

> +1 on EOL branch-1 and all branch-1.x.
>
> Regards,
> Pankaj
>
> On Thu, Apr 1, 2021 at 3:34 AM Andrew Purtell  wrote:
>
> > Is it time to consider EOL of branch-1 and all 1.x releases ?
> >
> > There doesn't seem to be much developer interest in branch-1 beyond
> > occasional maintenance. This is understandable. Per our compatibility
> > guidelines, branch-1 commits must be compatible with Java 7, and the
> range
> > of acceptable versions of third party dependencies is also restricted due
> > to Java 7 compatibility requirements. Most developers are writing code
> with
> > Java 8+ idioms these days. For that reason and because the branch-1 code
> > base is generally aged at this point, all but trivial (or lucky!)
> backports
> > require substantial changes in order to integrate adequately. Let me also
> > observe that branch-1 artifacts are not fully compatible with Java 11 or
> > later. (The shell is a good example of such issues: The version of
> > jruby-complete required by branch-1 is not compatible with Java 11 and
> > upgrading to the version used by branch-2 causes shell commands to error
> > out due to Ruby language changes.)
> >
> > We can a priori determine there is insufficient motivation for production
> > of release artifacts for the PMC to vote upon. Otherwise, someone would
> > have done it. We had 12 releases from branch-2 derived code in 2019, 13
> > releases from branch-2 derived code in 2020, and so far we have had 3
> > releases from branch-2 derived code in 2021. In contrast, we had 8
> releases
> > from branch-1 derived code in 2019, 0 releases from branch-1 in 2020, and
> > so far 0 releases from branch-1 in 2021.
> >
> > *  2021202020191.x0282.x31312*
> >
> > If there is someone interested in continuing branch-1, now is the time to
> > commit. However let me be clear that simply expressing an abstract desire
> > to see continued branch-1 releases will not be that useful. It will be
> > noted, but will not have much real world impact. Apache is a do-ocracy.
> In
> > the absence of intrinsic motivation of project participants, which is
> what
> > we seem to have here, you will need to do something: Fix the
> compatibility
> > issues, if any between the last release of 1.x and the current branch-1
> > head; fix any failing and flaky unit tests; produce release artifacts;
> and
> > submit those artifacts to the PMC for voting. Or, convince someone with
> > commit rights and/or PMC membership to undertake these actions on your
> > behalf.
> >
> > Otherwise, I respectfully submit for your consideration, it is time to
> > declare  branch-1 and all 1.x code lines EOL, simply acknowledging what
> has
> > effectively already happened.
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> >- A23, Crosstalk
> >
>

[jira] [Resolved] (HBASE-25721) Add 2.3.5 to the downloads page

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-25721.
--
Resolution: Fixed

> Add 2.3.5 to the downloads page
> ---
>
> Key: HBASE-25721
> URL: https://issues.apache.org/jira/browse/HBASE-25721
> Project: HBase
>  Issue Type: Task
>  Components: community
>        Reporter: Huaxiang Sun
>    Assignee: Huaxiang Sun
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-25722) Update reporter tool with new release, 2.3.5

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-25722:


 Summary: Update reporter tool with new release, 2.3.5
 Key: HBASE-25722
 URL: https://issues.apache.org/jira/browse/HBASE-25722
 Project: HBase
  Issue Type: Sub-task
Reporter: Huaxiang Sun
Assignee: Viraj Jasani


Reporter tool: [https://reporter.apache.org/addrelease.html?hbase]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-25721) Add 2.3.5 to the downloads page

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-25721:


 Summary: Add 2.3.5 to the downloads page
 Key: HBASE-25721
 URL: https://issues.apache.org/jira/browse/HBASE-25721
 Project: HBase
  Issue Type: Task
  Components: community
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] HBase 2.3.5 release candidate (RC1) is available

[Huaxiang Sun]

Thanks Pankaj!

Update:

With five (5) binding +1 votes, and two (2) nonbinding +1 vote, and no 0 or
-1 votes, this vote passes.

On Tue, Mar 30, 2021 at 10:27 AM Pankaj Kumar 
wrote:

> I'm bit late to vote (was waiting for the unit test result) :-)
>
>
> +1 (non-binding)
>
>  Below the output from the vote script,
> * Signature: ok
> * Checksum : ok
> * Rat check (11.0.10): ok
>  - mvn clean apache-rat:check -D hadoop.profile=3.0
> * Built from source (11.0.10): ok
>  - mvn clean install -D hadoop.profile=3.0 -DskipTests
> * Unit tests pass (11.0.10): ok
>  - mvn package -P runSmallTests -D hadoop.profile=3.0
> -Dsurefire.rerunFailingTestsCount=3
>
> * Installed a single node cluster and exercised basic commands.
>
> Regards,
> Pankaj
>
> On Tue, Mar 30, 2021 at 10:20 PM Huaxiang Sun 
> wrote:
>
> > With five (5) binding +1 votes, and one (1) nonbinding +1
> > vote, and no 0 or -1 votes, this vote passes.
> >
> > Thanks to all who voted on this release candidate.
> >
> > On Mon, Mar 29, 2021 at 5:50 PM Andrew Purtell 
> > wrote:
> >
> > > +1 (binding)
> > >
> > > * Signature: ok
> > > * Checksum : ok
> > > * Rat check (1.8.0_292): ok
> > >  - mvn clean apache-rat:check
> > > * Built from source (1.8.0_292): ok
> > >  - mvn clean install -DskipTests
> > > * Unit tests pass (1.8.0_292): ok
> > >
> > >
> > >
> > > On Fri, Mar 26, 2021 at 12:10 PM Huaxiang Sun 
> > > wrote:
> > >
> > > > Please vote on this Apache hbase release candidate,
> > > >
> > > > hbase-2.3.5RC1
> > > >
> > > >
> > > > The VOTE will remain open for at least 72 hours.
> > > >
> > > >
> > > > [ ] +1 Release this package as Apache hbase 2.3.5
> > > >
> > > > [ ] -1 Do not release this package because ...
> > > >
> > > >
> > > > The tag to be voted on is 2.3.5RC1:
> > > >
> > > >
> > > >   https://github.com/apache/hbase/tree/2.3.5RC1
> > > >
> > > >
> > > > This tag currently points to git reference
> > > >
> > > >
> > > >   fd3fdc08d1cd43eb3432a1a70d31c3aece6ecabe
> > > >
> > > >
> > > > The release files, including signatures, digests, as well as
> CHANGES.md
> > > >
> > > > and RELEASENOTES.md included in this RC can be found at:
> > > >
> > > >
> > > >   https://dist.apache.org/repos/dist/dev/hbase/2.3.5RC1/
> > > >
> > > >
> > > > Maven artifacts are available in a staging repository at:
> > > >
> > > >
> > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehbase-1439/
> > > >
> > > >
> > > > Artifacts were signed with the 0x117C835E key which can be found in:
> > > >
> > > >
> > > >   https://dist.apache.org/repos/dist/release/hbase/KEYS
> > > >
> > > >
> > > > To learn more about Apache hbase, please see
> > > >
> > > >
> > > >   http://hbase.apache.org/
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Your HBase Release Manager
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew
> > >
> > > Words like orphans lost among the crosstalk, meaning torn from truth's
> > > decrepit hands
> > >- A23, Crosstalk
> > >
> >
>

Re: [VOTE] HBase 2.3.5 release candidate (RC1) is available

[Huaxiang Sun]

With five (5) binding +1 votes, and one (1) nonbinding +1
vote, and no 0 or -1 votes, this vote passes.

Thanks to all who voted on this release candidate.

On Mon, Mar 29, 2021 at 5:50 PM Andrew Purtell  wrote:

> +1 (binding)
>
> * Signature: ok
> * Checksum : ok
> * Rat check (1.8.0_292): ok
>  - mvn clean apache-rat:check
> * Built from source (1.8.0_292): ok
>  - mvn clean install -DskipTests
> * Unit tests pass (1.8.0_292): ok
>
>
>
> On Fri, Mar 26, 2021 at 12:10 PM Huaxiang Sun 
> wrote:
>
> > Please vote on this Apache hbase release candidate,
> >
> > hbase-2.3.5RC1
> >
> >
> > The VOTE will remain open for at least 72 hours.
> >
> >
> > [ ] +1 Release this package as Apache hbase 2.3.5
> >
> > [ ] -1 Do not release this package because ...
> >
> >
> > The tag to be voted on is 2.3.5RC1:
> >
> >
> >   https://github.com/apache/hbase/tree/2.3.5RC1
> >
> >
> > This tag currently points to git reference
> >
> >
> >   fd3fdc08d1cd43eb3432a1a70d31c3aece6ecabe
> >
> >
> > The release files, including signatures, digests, as well as CHANGES.md
> >
> > and RELEASENOTES.md included in this RC can be found at:
> >
> >
> >   https://dist.apache.org/repos/dist/dev/hbase/2.3.5RC1/
> >
> >
> > Maven artifacts are available in a staging repository at:
> >
> >
> >
> https://repository.apache.org/content/repositories/orgapachehbase-1439/
> >
> >
> > Artifacts were signed with the 0x117C835E key which can be found in:
> >
> >
> >   https://dist.apache.org/repos/dist/release/hbase/KEYS
> >
> >
> > To learn more about Apache hbase, please see
> >
> >
> >   http://hbase.apache.org/
> >
> >
> > Thanks,
> >
> > Your HBase Release Manager
> >
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>- A23, Crosstalk
>

Re: [VOTE] HBase 2.3.5 release candidate (RC1) is available

[Huaxiang Sun]

+ 1 (non-binding)

* Signature: ok

* Checksum : ok

* Rat check (1.8.0_242): ok

 - mvn clean apache-rat:check

* Built from source (1.8.0_242): ok

 - mvn clean install  -DskipTests

* Unit tests pass (1.8.0_242): ok

 - mvn package -P runSmallTests  -Dsurefire.rerunFailingTestsCount=3


Put the release at a 15 node cluster (3 masters, 12 region servers). Run
ITBLL to insert 3 Billions' rows, it succeeded.

[VOTE] HBase 2.3.5 release candidate (RC1) is available

[Huaxiang Sun]

Please vote on this Apache hbase release candidate,

hbase-2.3.5RC1


The VOTE will remain open for at least 72 hours.


[ ] +1 Release this package as Apache hbase 2.3.5

[ ] -1 Do not release this package because ...


The tag to be voted on is 2.3.5RC1:


  https://github.com/apache/hbase/tree/2.3.5RC1


This tag currently points to git reference


  fd3fdc08d1cd43eb3432a1a70d31c3aece6ecabe


The release files, including signatures, digests, as well as CHANGES.md

and RELEASENOTES.md included in this RC can be found at:


  https://dist.apache.org/repos/dist/dev/hbase/2.3.5RC1/


Maven artifacts are available in a staging repository at:


  https://repository.apache.org/content/repositories/orgapachehbase-1439/


Artifacts were signed with the 0x117C835E key which can be found in:


  https://dist.apache.org/repos/dist/release/hbase/KEYS


To learn more about Apache hbase, please see


  http://hbase.apache.org/


Thanks,

Your HBase Release Manager

[jira] [Resolved] (HBASE-25590) Bulkload replication HFileRefs cannot be cleared in some cases where set exclude-namespace/exclude-table-cfs

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-25590.
--
Resolution: Fixed

Resolving it for 2.3.5 release. Please reopen when landing the 2.2 patch.

> Bulkload replication HFileRefs cannot be cleared in some cases where set 
> exclude-namespace/exclude-table-cfs
> 
>
> Key: HBASE-25590
> URL: https://issues.apache.org/jira/browse/HBASE-25590
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Affects Versions: 3.0.0-alpha-1, 2.2.6, 2.3.4, 2.4.1
>Reporter: Sun Xin
>Assignee: Sun Xin
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.5
>
>
> In 
> [ReplicationSource#addHFileRefs|https://github.com/apache/hbase/blob/ed90a14995acd87111d2b9849f07d84418ca43d4/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java#L264],
>  we may add unwanted hfiles to the _HFileRefs_ if a peer is set 
> _replicate_all_ true and set _exclude-namespace/exclude-table-cfs_.
> These unwanted _HFileRefs_ will not be replicated to remote cluster and not 
> be cleared.
> Two problems are caused by this bug:
>  # The metric sizeOfHFileRefsQueue cannot be zeroed.
>  # Referenced HFiles cannot be deleted by _ReplicationHFileCleaner._



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-25691) Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-25691.
--
Resolution: Fixed

> Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile
> 
>
> Key: HBASE-25691
> URL: https://issues.apache.org/jira/browse/HBASE-25691
> Project: HBase
>  Issue Type: Test
>  Components: test
>Affects Versions: 2.3.4
>    Reporter: Huaxiang Sun
>    Assignee: Huaxiang Sun
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.5, 2.4.3
>
>
> Saw this test failure from 2.3 nightly.
> https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/190/testReport/junit/org.apache.hadoop.hbase.io.hfile.bucket/TestVerifyBucketCacheFile/health_checks___yetus_jdk8_hadoop2_checks___testRetrieveFromFile_1__blockSize_16_384__bucketSizes__I_371a67ec_/
> h1. Regression
> health checks / yetus jdk8 hadoop2 checks / 
> org.apache.hadoop.hbase.io.hfile.bucket.TestVerifyBucketCacheFile.testRetrieveFromFile[1:
>  blockSize=16,384, bucketSizes=[I@371a67ec]
> Failing for the past 1 build (Since 
> [!https://ci-hadoop.apache.org/static/e247241e/images/16x16/red.png! 
> #190|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/190/]
>  )
> [Took 0.32 
> sec.|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/190/testReport/junit/org.apache.hadoop.hbase.io.hfile.bucket/TestVerifyBucketCacheFile/health_checks___yetus_jdk8_hadoop2_checks___testRetrieveFromFile_1__blockSize_16_384__bucketSizes__I_371a67ec_/history]
>  
> h3. Stacktrace
> java.lang.AssertionError at 
> org.apache.hadoop.hbase.io.hfile.bucket.TestVerifyBucketCacheFile.testRetrieveFromFile(TestVerifyBucketCacheFile.java:136)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-25691) Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-25691:


 Summary: Test failure: 
TestVerifyBucketCacheFile.testRetrieveFromFile
 Key: HBASE-25691
 URL: https://issues.apache.org/jira/browse/HBASE-25691
 Project: HBase
  Issue Type: Test
  Components: test
Affects Versions: 2.3.4
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun


Saw this test failure from 2.3 nightly.

https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/190/testReport/junit/org.apache.hadoop.hbase.io.hfile.bucket/TestVerifyBucketCacheFile/health_checks___yetus_jdk8_hadoop2_checks___testRetrieveFromFile_1__blockSize_16_384__bucketSizes__I_371a67ec_/
h1. Regression

health checks / yetus jdk8 hadoop2 checks / 
org.apache.hadoop.hbase.io.hfile.bucket.TestVerifyBucketCacheFile.testRetrieveFromFile[1:
 blockSize=16,384, bucketSizes=[I@371a67ec]
Failing for the past 1 build (Since 
[!https://ci-hadoop.apache.org/static/e247241e/images/16x16/red.png! 
#190|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/190/]
 )
[Took 0.32 
sec.|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/190/testReport/junit/org.apache.hadoop.hbase.io.hfile.bucket/TestVerifyBucketCacheFile/health_checks___yetus_jdk8_hadoop2_checks___testRetrieveFromFile_1__blockSize_16_384__bucketSizes__I_371a67ec_/history]
 
h3. Stacktrace

java.lang.AssertionError at 
org.apache.hadoop.hbase.io.hfile.bucket.TestVerifyBucketCacheFile.testRetrieveFromFile(TestVerifyBucketCacheFile.java:136)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Time to 2.3.5 (was: Delaying 2.3.5 another month)

[Huaxiang Sun]

Thanks Nick. I am starting the release process today.

Huaxiang

On Wed, Mar 17, 2021 at 10:28 AM Nick Dimiduk  wrote:

> Hi Everyone,
>
> Looks like we have a nice 40-ish commits on branch-2.3, so I think it's
> time for another release. Huaxiang has again volunteered to run this
> release, so I will defer 2.3.5 to him. As for timing, I think it's best if
> we let the current 2.4 release complete (looks like it's close). Please
> speak up if you have any nice patches you're ready to land, we'll see about
> their inclusion.
>
> Thank you, Huaxiang!
>
> Thanks,
> Nick
>
> On Fri, Feb 26, 2021 at 10:31 AM Nick Dimiduk  wrote:
>
> > Heya team,
> >
> > There are fewer than 20 issues resolved against the head of branch-2.3
> and
> > none of them are marked as Critical. Thus I think we can postpone the
> next
> > 2.3 release by another month. If you have concerns or disagree, please
> > reply here to let me know.
> >
> > Thanks,
> > Nick
> >
>

[jira] [Resolved] (HBASE-25639) meta replica state is not respected during active master switch

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-25639.
--
Fix Version/s: 2.3.5
   Resolution: Fixed

> meta replica state is not respected during active master switch
> ---
>
> Key: HBASE-25639
> URL: https://issues.apache.org/jira/browse/HBASE-25639
> Project: HBase
>  Issue Type: Bug
>  Components: meta replicas
>Affects Versions: 2.0.6, 2.1.9, 2.2.6, 2.3.4
>    Reporter: Huaxiang Sun
>    Assignee: Huaxiang Sun
>Priority: Critical
> Fix For: 2.3.5
>
>
> We saw this warning in master log.
> WARN org.apache.hadoop.hbase.master.assignment.AssignmentManager: No 
> RegionStateNode for hbase:meta,,1_0003 but reported as up on 
> server1.example.com,16020,1614958467735; closing...
>  
> The root cause is that meta replica regions are in zookeeper, and these state 
> are not iterated by the new active master so it loses track.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

2.3.5 RC next week

[Huaxiang Sun]

There will be a RC for 2.3.5 available for voting next week.

If you have any pending work and need assistance to get it committed please
let me know.

[jira] [Created] (HBASE-25640) Support hbase rpc compression for remote rpc only

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-25640:


 Summary: Support hbase rpc compression for remote rpc only
 Key: HBASE-25640
 URL: https://issues.apache.org/jira/browse/HBASE-25640
 Project: HBase
  Issue Type: Improvement
  Components: rpc
Affects Versions: 2.3.4
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun


The purpose of Rpc compression is to save network bandwidth. For local 
communication (both hbase client and RS are on the same node), rpc compression 
is unnecessary as local communication is memory copy only and does not go 
through nic. Rpc compression for local communication will be a waste of cpu 
computation power as compress/decompress is cpu intensive. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-25639) meta replica state is not respected during active master switch

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-25639:


 Summary: meta replica state is not respected during active master 
switch
 Key: HBASE-25639
 URL: https://issues.apache.org/jira/browse/HBASE-25639
 Project: HBase
  Issue Type: Bug
  Components: meta replicas
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun


We saw this warning in master log.

WARN org.apache.hadoop.hbase.master.assignment.AssignmentManager: No 
RegionStateNode for hbase:meta,,1_0003 but reported as up on 
server1.example.com,16020,1614958467735; closing...

 

The root cause is that meta replica regions are in zookeeper, and these state 
are not iterated by the new active master so it loses track.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-25537) Misleading Range metrcis

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-25537:


 Summary: Misleading Range metrcis 
 Key: HBASE-25537
 URL: https://issues.apache.org/jira/browse/HBASE-25537
 Project: HBase
  Issue Type: Bug
  Components: metrics
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun
 Fix For: 2.3.4
 Attachments: Screen Shot 2021-01-27 at 1.09.32 PM.png

Found some cases that max value is included in a smaller range, which is 
confusing. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-25417) Send announce email

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-25417.
--
Resolution: Fixed

> Send announce email
> ---
>
> Key: HBASE-25417
> URL: https://issues.apache.org/jira/browse/HBASE-25417
> Project: HBase
>  Issue Type: Sub-task
>        Reporter: Huaxiang Sun
>        Assignee: Huaxiang Sun
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-25409) Release 2.3.4

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-25409.
--
Resolution: Fixed

> Release 2.3.4
> -
>
> Key: HBASE-25409
> URL: https://issues.apache.org/jira/browse/HBASE-25409
> Project: HBase
>  Issue Type: Task
>  Components: community
>        Reporter: Huaxiang Sun
>    Assignee: Huaxiang Sun
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[ANNOUNCE] Apache HBase 2.3.4 is now available for download

[Huaxiang Sun]

The HBase team is happy to announce the immediate availability of HBase

2.3.4.


Apache HBase™ is an open-source, distributed, versioned, non-relational

database.

Apache HBase gives you low latency random access to billions of rows with

millions of columns atop non-specialized hardware. To learn more about

HBase,

see https://hbase.apache.org/.


HBase 2.3.4 is the fourth patch release in the HBase 2.3.x line, which aims

to

improve the stability and reliability of HBase. This release includes 62 bug

fixes and improvements since 2.3.3.


The full list of issues and release notes can be found here:

CHANGELOG: https://downloads.apache.org/hbase/2.3.4/CHANGES.md

RELEASENOTES: https://downloads.apache.org/hbase/2.3.4/RELEASENOTES.md


or via our issue tracker:

https://issues.apache.org/jira/projects/HBASE/versions/12349309


To download please follow the links and instructions on our website:


https://hbase.apache.org/downloads.html



Questions, comments, and problems are always welcome at:

dev@hbase.apache.org

u...@hbase.apache.org


Thanks to all who contributed and made this release possible.


Cheers,

The HBase Dev Team

[jira] [Resolved] (HBASE-25416) Add 2.3.4 to the downloads page

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-25416.
--
Fix Version/s: (was: 3.0.0-alpha-1)
   Resolution: Fixed

> Add 2.3.4 to the downloads page
> ---
>
> Key: HBASE-25416
> URL: https://issues.apache.org/jira/browse/HBASE-25416
> Project: HBase
>  Issue Type: Sub-task
>        Reporter: Huaxiang Sun
>        Assignee: Huaxiang Sun
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-25412) Release version 2.3.4 in Jira

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-25412.
--
Resolution: Fixed

> Release version 2.3.4 in Jira
> -
>
> Key: HBASE-25412
> URL: https://issues.apache.org/jira/browse/HBASE-25412
> Project: HBase
>  Issue Type: Sub-task
>        Reporter: Huaxiang Sun
>        Assignee: Huaxiang Sun
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-25411) "Release" staged nexus repository

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-25411.
--
Resolution: Fixed

> "Release" staged nexus repository
> -
>
> Key: HBASE-25411
> URL: https://issues.apache.org/jira/browse/HBASE-25411
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-25410) Spin RCs

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-25410.
--
Resolution: Fixed

> Spin RCs
> 
>
> Key: HBASE-25410
> URL: https://issues.apache.org/jira/browse/HBASE-25410
> Project: HBase
>  Issue Type: Sub-task
>        Reporter: Huaxiang Sun
>        Assignee: Huaxiang Sun
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-25413) Promote 2.3.4 RC artifacts in svn

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-25413.
--
Resolution: Fixed

> Promote 2.3.4 RC artifacts in svn
> -
>
> Key: HBASE-25413
> URL: https://issues.apache.org/jira/browse/HBASE-25413
> Project: HBase
>  Issue Type: Sub-task
>        Reporter: Huaxiang Sun
>        Assignee: Huaxiang Sun
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-25415) Push signed release tag

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-25415.
--
Resolution: Fixed

> Push signed release tag
> ---
>
> Key: HBASE-25415
> URL: https://issues.apache.org/jira/browse/HBASE-25415
> Project: HBase
>  Issue Type: Sub-task
>        Reporter: Huaxiang Sun
>        Assignee: Huaxiang Sun
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] HBase 2.3.4 release candidate (RC4) is available

[Huaxiang Sun]

With four (4) binding +1 votes, and one (1) nonbinding +1
vote, and no 0 or -1 votes, this vote passes.

Thanks to all who voted on the release candidate.

On Thu, Jan 21, 2021 at 11:55 AM Huaxiang Sun  wrote:

> We have enough affirmative votes (4 binding +1s) to close this vote and
> the vote is open for over 72 hours.
> If there are any pending tests/votes please let me know. Otherwise I will
> be closing this vote tomorrow morning Pacific time.
>
> On Wed, Jan 20, 2021 at 9:31 PM Stack  wrote:
>
>> +1 (binding)
>>
>>  * Signature: good
>>  * Checksum : good
>>  * Built from src:  mvn clean install -DskipTests assembly:single
>> -Dhadoop.profile=3.0 -Prelease
>>  * Checked UI
>>  * Loaded 3m w/ pe running local MR
>>  * Verified loaded 3m w/ MR and w/o MR
>>  * Poked about in the shell
>>
>> S
>>
>>
>> On Wed, Jan 20, 2021 at 11:20 AM Nick Dimiduk 
>> wrote:
>>
>> > +1
>> >
>> > * Signature: ok
>> > * Checksum : ok
>> > * Rat check (1.8.0_275): ok
>> >  - mvn clean apache-rat:check -D skipTests
>> > * Rat check (11.0.9.1): ok
>> >  - mvn clean apache-rat:check -D skipTests -D hadoop.profile=3.0
>> > * Built from source (1.8.0_275): ok
>> >  - mvn clean install -D skipTests -DskipTests
>> > * Built from source (11.0.9.1): ok
>> >  - mvn clean install -D skipTests -D hadoop.profile=3.0
>> -DskipTests
>> > * Unit tests pass (1.8.0_275): ok
>> >  - mvn package -P runAllTests -D skipTests
>> > -Dsurefire.rerunFailingTestsCount=3
>> > * Unit tests pass (11.0.9.1): ok
>> >  - mvn package -P runAllTests -D skipTests -D hadoop.profile=3.0
>> > -Dsurefire.rerunFailingTestsCount=3
>> > * bin tarball launches on JDK 11.0.8+10.
>> > * Revision number on webUI matches tag in git.
>> > * Jenkins builds have been broken lately, but Viraj noted the
>> WIP
>> > fix.
>> >
>> > On Thu, Jan 14, 2021 at 7:04 PM Huaxiang Sun 
>> > wrote:
>> >
>> > > Please vote on this Apache hbase release candidate,
>> > >
>> > > hbase-2.3.4RC4
>> > >
>> > >
>> > > The VOTE will remain open for at least 72 hours.
>> > >
>> > >
>> > > [ ] +1 Release this package as Apache hbase 2.3.4
>> > >
>> > > [ ] -1 Do not release this package because ...
>> > >
>> > >
>> > > The tag to be voted on is 2.3.4RC4:
>> > >
>> > >
>> > >   https://github.com/apache/hbase/tree/2.3.4RC4
>> > >
>> > >
>> > > The release files, including signatures, digests, as well as
>> CHANGES.md
>> > >
>> > > and RELEASENOTES.md included in this RC can be found at:
>> > >
>> > >
>> > >   https://dist.apache.org/repos/dist/dev/hbase/2.3.4RC4/
>> > >
>> > >
>> > > Maven artifacts are available in a staging repository at:
>> > >
>> > >
>> > >
>> > https://repository.apache.org/content/repositories/orgapachehbase-1433/
>> > >
>> > >
>> > > Artifacts were signed with the huaxiang...@apache.org key which can
>> be
>> > > found in:
>> > >
>> > >
>> > >   https://dist.apache.org/repos/dist/release/hbase/KEYS
>> > >
>> > >
>> > > To learn more about Apache hbase, please see
>> > >
>> > >
>> > >   http://hbase.apache.org/
>> > >
>> > >
>> > > Thanks,
>> > >
>> > > Your HBase Release Manager
>> > >
>> >
>>
>

Re: [VOTE] HBase 2.3.4 release candidate (RC4) is available

[Huaxiang Sun]

We have enough affirmative votes (4 binding +1s) to close this vote and the
vote is open for over 72 hours.
If there are any pending tests/votes please let me know. Otherwise I will
be closing this vote tomorrow morning Pacific time.

On Wed, Jan 20, 2021 at 9:31 PM Stack  wrote:

> +1 (binding)
>
>  * Signature: good
>  * Checksum : good
>  * Built from src:  mvn clean install -DskipTests assembly:single
> -Dhadoop.profile=3.0 -Prelease
>  * Checked UI
>  * Loaded 3m w/ pe running local MR
>  * Verified loaded 3m w/ MR and w/o MR
>  * Poked about in the shell
>
> S
>
>
> On Wed, Jan 20, 2021 at 11:20 AM Nick Dimiduk  wrote:
>
> > +1
> >
> > * Signature: ok
> > * Checksum : ok
> > * Rat check (1.8.0_275): ok
> >  - mvn clean apache-rat:check -D skipTests
> > * Rat check (11.0.9.1): ok
> >  - mvn clean apache-rat:check -D skipTests -D hadoop.profile=3.0
> > * Built from source (1.8.0_275): ok
> >  - mvn clean install -D skipTests -DskipTests
> > * Built from source (11.0.9.1): ok
> >  - mvn clean install -D skipTests -D hadoop.profile=3.0
> -DskipTests
> > * Unit tests pass (1.8.0_275): ok
> >  - mvn package -P runAllTests -D skipTests
> > -Dsurefire.rerunFailingTestsCount=3
> > * Unit tests pass (11.0.9.1): ok
> >  - mvn package -P runAllTests -D skipTests -D hadoop.profile=3.0
> > -Dsurefire.rerunFailingTestsCount=3
> > * bin tarball launches on JDK 11.0.8+10.
> > * Revision number on webUI matches tag in git.
> > * Jenkins builds have been broken lately, but Viraj noted the WIP
> > fix.
> >
> > On Thu, Jan 14, 2021 at 7:04 PM Huaxiang Sun 
> > wrote:
> >
> > > Please vote on this Apache hbase release candidate,
> > >
> > > hbase-2.3.4RC4
> > >
> > >
> > > The VOTE will remain open for at least 72 hours.
> > >
> > >
> > > [ ] +1 Release this package as Apache hbase 2.3.4
> > >
> > > [ ] -1 Do not release this package because ...
> > >
> > >
> > > The tag to be voted on is 2.3.4RC4:
> > >
> > >
> > >   https://github.com/apache/hbase/tree/2.3.4RC4
> > >
> > >
> > > The release files, including signatures, digests, as well as CHANGES.md
> > >
> > > and RELEASENOTES.md included in this RC can be found at:
> > >
> > >
> > >   https://dist.apache.org/repos/dist/dev/hbase/2.3.4RC4/
> > >
> > >
> > > Maven artifacts are available in a staging repository at:
> > >
> > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehbase-1433/
> > >
> > >
> > > Artifacts were signed with the huaxiang...@apache.org key which can be
> > > found in:
> > >
> > >
> > >   https://dist.apache.org/repos/dist/release/hbase/KEYS
> > >
> > >
> > > To learn more about Apache hbase, please see
> > >
> > >
> > >   http://hbase.apache.org/
> > >
> > >
> > > Thanks,
> > >
> > > Your HBase Release Manager
> > >
> >
>

[jira] [Resolved] (HBASE-25368) Filter out more invalid encoded name in isEncodedRegionName(byte[] regionName)

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-25368.
--
Fix Version/s: 3.0.0-alpha-1
   Resolution: Fixed

> Filter out more invalid encoded name in isEncodedRegionName(byte[] 
> regionName) 
> ---
>
> Key: HBASE-25368
> URL: https://issues.apache.org/jira/browse/HBASE-25368
> Project: HBase
>  Issue Type: Improvement
>  Components: Client
>    Reporter: Huaxiang Sun
>    Assignee: Huaxiang Sun
>Priority: Major
> Fix For: 3.0.0-alpha-1
>
>
> {code:java}
> public static boolean isEncodedRegionName(byte[] regionName) {
>   // If not parseable as region name, presume encoded. TODO: add stringency; 
> e.g. if hex.
>   return parseRegionNameOrReturnNull(regionName) == null && regionName.length 
> <= MD5_HEX_LENGTH;
> }
> Right now, if it passes in an table name, it still thinks  it is a encoded 
> region name and will result in unnecessary registry query for meta regions. 
> This can be avoided if table names can be filtered out early in this 
> method.{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] HBase 2.3.4 release candidate (RC4) is available

[Huaxiang Sun]

Hi Folks,
The 2.3.4RC4 vote has been up for 4 days, so far, we got 1 +1
(binding), 1 +1 (non-binding). When you have a moment, please vote.

Thanks
Your HBase Release Manager

On Sun, Jan 17, 2021 at 10:08 PM Huaxiang Sun  wrote:

> >Put 2.4.0 to a 15 node cluster (3 masters, 12 region servers), run itbll
> with serverKilling chaos monkey.
>
> > Added 3 billions row, it’ll finished successfully (verified).
>
>
> A typo, I put 2.3.4RC4 to the cluster and itbll run successfully.
>
> On Sat, Jan 16, 2021 at 10:45 PM Viraj Jasani  wrote:
>
>> +1 (binding)
>>
>> * Signature: ok
>> * Checksum : ok
>> * Rat check (1.8.0_171): ok
>>  - mvn clean apache-rat:check
>> * Built from source (1.8.0_171): ok
>>  - mvn clean install -DskipTests
>> * Unit tests pass (1.8.0_171): ok
>>  - mvn package -P runSmallTests
>> * All tests: branch-2.3 nightly runs look good
>> (one consistent failure with TestSplitWALManager which is not
>> part of this RC and the work is in progress to resolve it)
>> * CHANGELOG: ok
>> * Release Notes: ok
>> * CRUD: ok
>>
>>
>> On 2021/01/15 03:03:47, Huaxiang Sun  wrote:
>> > Please vote on this Apache hbase release candidate,
>> >
>> > hbase-2.3.4RC4
>> >
>> >
>> > The VOTE will remain open for at least 72 hours.
>> >
>> >
>> > [ ] +1 Release this package as Apache hbase 2.3.4
>> >
>> > [ ] -1 Do not release this package because ...
>> >
>> >
>> > The tag to be voted on is 2.3.4RC4:
>> >
>> >
>> >   https://github.com/apache/hbase/tree/2.3.4RC4
>> >
>> >
>> > The release files, including signatures, digests, as well as CHANGES.md
>> >
>> > and RELEASENOTES.md included in this RC can be found at:
>> >
>> >
>> >   https://dist.apache.org/repos/dist/dev/hbase/2.3.4RC4/
>> >
>> >
>> > Maven artifacts are available in a staging repository at:
>> >
>> >
>> >
>> https://repository.apache.org/content/repositories/orgapachehbase-1433/
>> >
>> >
>> > Artifacts were signed with the huaxiang...@apache.org key which can be
>> > found in:
>> >
>> >
>> >   https://dist.apache.org/repos/dist/release/hbase/KEYS
>> >
>> >
>> > To learn more about Apache hbase, please see
>> >
>> >
>> >   http://hbase.apache.org/
>> >
>> >
>> > Thanks,
>> >
>> > Your HBase Release Manager
>> >
>>
>