Re: [VOTE] Release candidate for HBase 2.5.0 (RC1) is available

2022-08-26 Thread Andrew Purtell 
>
org/apache/hadoop/hbase/io/encoding/DataBlockEncoding.createEncoder:(Ljava/lang/String;)Lorg/apache/hadoop/hbase/io/encoding/DataBlockEncoder
Access level has been changed from protected to package-private.

This was the essence of the error-prone warning. It is a code smell to have
protected methods in final classes.

On Fri, Aug 26, 2022 at 12:52 PM Viraj Jasani  wrote:

> +0
>
> * Signature: ok
> * Checksum : ok
> * Rat check (1.8.0_341): ok
>  - mvn clean apache-rat:check
> * Built from source (1.8.0_341): ok
>  - mvn clean install  -DskipTests
> * Unit tests pass (1.8.0_341): failed (TestUsersOperationsWithSecureHadoop
> has known failure locally)
>  - mvn package -P runSmallTests  -Dsurefire.rerunFailingTestsCount=3
> * A bit of high load testing locally looks good
>
> The compat report has few high sev problems reported but not all need to be
> taken care of given that majority are affecting deprecated/private APIs.
> However, there seems one high sev problem that we might want to consider
> fixing:
>
>
> org/apache/hadoop/hbase/io/encoding/DataBlockEncoding.createEncoder:(Ljava/lang/String;)Lorg/apache/hadoop/hbase/io/encoding/DataBlockEncoder
> Access level has been changed from protected to package-private.
>
> This was done as part of the recent error-prone findings. Either we can
> bring back protected access or, we already have public getter
> *getEncoder()*,
> hence I think perhaps we can make *createEncoder(String
> fullyQualifiedClassName)* as IA.Private. Either way is fine.
>
> Functional testing seems good, thanks Nick.
>
>
> On Wed, Aug 24, 2022 at 2:53 AM Nick Dimiduk  wrote:
>
> > Please vote on this Apache hbase release candidate,
> > hbase-2.5.0RC1
> >
> > The VOTE will remain open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache hbase 2.5.0
> > [ ] -1 Do not release this package because ...
> >
> > The tag to be voted on is 2.5.0RC1:
> >
> >   https://github.com/apache/hbase/tree/2.5.0RC1
> >
> > This tag currently points to git reference
> >
> >   2ecd8bd6d615ca49bfb329b3c0c126c80846d4ab
> >
> > The release files, including signatures, digests, as well as CHANGES.md
> > and RELEASENOTES.md included in this RC can be found at:
> >
> >   https://dist.apache.org/repos/dist/dev/hbase/2.5.0RC1/
> >
> > Maven artifacts are available in a staging repository at:
> >
> >
> https://repository.apache.org/content/repositories/orgapachehbase-1494/
> >
> > Artifacts were signed with the 0x18567F39 key which can be found in:
> >
> >   https://downloads.apache.org/hbase/KEYS
> >
> > To learn more about Apache hbase, please see
> >
> >   http://hbase.apache.org/
> >
> > Thanks,
> > Your HBase Release Manager
> >
>


-- 
Best regards,
Andrew

Unrest, ignorance distilled, nihilistic imbeciles -
It's what we’ve earned
Welcome, apocalypse, what’s taken you so long?
Bring us the fitting end that we’ve been counting on
   - A23, Welcome, Apocalypse

Re: [VOTE] Release candidate for HBase 2.5.0 (RC1) is available

2022-08-26 Thread Stack 
+1

On amd64 bash -x ./dev-support/hbase-vote.sh --source
https://dist.apache.org/repos/dist/dev/hbase/2.5.0RC1/ --output-dir
~/Downloads/250rc1 -P runSmallTests -D hadoop.profile=3.0

* Signature: ok
* Checksum : ok
* Rat check (11.0.12): ok
 - mvn clean apache-rat:check -D hadoop.profile=3.0
* Built from source (11.0.12): ok
 - mvn clean install -D hadoop.profile=3.0 -DskipTests
* Unit tests pass (11.0.12): failed
 - mvn package -P runSmallTests -D hadoop.profile=3.0
-Dsurefire.rerunFailingTestsCount=3

On arm64  bash -x ./dev-support/hbase-vote.sh --source
https://dist.apache.org/repos/dist/dev/hbase/2.5.0RC1/ --output-dir
~/Downloads/250rc1 -D skipTests -D hadoop.profile=3.0

* Signature: ok
* Checksum : failed
* Rat check (11.0.15): ok
 - mvn clean apache-rat:check -D skipTests -D hadoop.profile=3.0
* Built from source (11.0.15): ok
 - mvn clean install -D skipTests -D hadoop.profile=3.0 -DskipTests
* Unit tests pass (11.0.15): ok
 - mvn package -P runAllTests -D skipTests -D hadoop.profile=3.0
-Dsurefire.rerunFailingTestsCount=3

(Didn't runSmallTests on M1/arm because fails with HBASE-27338 brotli
compression lib tests fail on arm64
)

* Looked at the compatibility report. The red-flagged items seem internals
or items unlikely to have been sub-classed by a downstreamer
(ColumnFamilyDescriptor).
* Ran ITBLL on 6-node "cluster" on jdk17. 25 clients doing 1M each (Took
some time to get a working build... it looks like I had to specify
hadoop.version and hadoop-three.version building: TBD).
* Ran 25 cilents doing 10M each. Some kills. Stuff came back fine and
promptly. Canary works.
* UI looks fine. Poked around. Fancy region visualizer doo-hickey!
* CLI works.

S




On Wed, Aug 24, 2022 at 2:53 AM Nick Dimiduk  wrote:

> Please vote on this Apache hbase release candidate,
> hbase-2.5.0RC1
>
> The VOTE will remain open for at least 72 hours.
>
> [ ] +1 Release this package as Apache hbase 2.5.0
> [ ] -1 Do not release this package because ...
>
> The tag to be voted on is 2.5.0RC1:
>
>   https://github.com/apache/hbase/tree/2.5.0RC1
>
> This tag currently points to git reference
>
>   2ecd8bd6d615ca49bfb329b3c0c126c80846d4ab
>
> The release files, including signatures, digests, as well as CHANGES.md
> and RELEASENOTES.md included in this RC can be found at:
>
>   https://dist.apache.org/repos/dist/dev/hbase/2.5.0RC1/
>
> Maven artifacts are available in a staging repository at:
>
>   https://repository.apache.org/content/repositories/orgapachehbase-1494/
>
> Artifacts were signed with the 0x18567F39 key which can be found in:
>
>   https://downloads.apache.org/hbase/KEYS
>
> To learn more about Apache hbase, please see
>
>   http://hbase.apache.org/
>
> Thanks,
> Your HBase Release Manager
>

Re: [VOTE] Release candidate for HBase 2.5.0 (RC1) is available

2022-08-26 Thread Huaxiang Sun 
+1

~/work/hbase-hs/hbase-1

* Signature: ok

* Checksum : failed

* Rat check (1.8.0_242): ok

 - mvn clean apache-rat:check

* Built from source (1.8.0_242): ok

 - mvn clean install  -DskipTests

* Unit tests pass (1.8.0_242): ok

 - mvn package -P runSmallTests  -Dsurefire.rerunFailingTestsCount=3

$ ./dev-support/hbase-vote.sh -s
https://dist.apache.org/repos/dist/dev/hbase/2.5.0RC1/ -P runSmallTests -o
~/hbase-2.5.0-vote


Also run


./bin/hbase ltt -num_keys 200 -write 10:2000:16 -num_tables 3


twice on a local cluster to trigger region split and compaction. It
completed successfully.

Also run into Region Visualizer's undefined region server issue.


Huaxiang



On Fri, Aug 26, 2022 at 12:52 PM Viraj Jasani  wrote:

> +0
>
> * Signature: ok
> * Checksum : ok
> * Rat check (1.8.0_341): ok
>  - mvn clean apache-rat:check
> * Built from source (1.8.0_341): ok
>  - mvn clean install  -DskipTests
> * Unit tests pass (1.8.0_341): failed (TestUsersOperationsWithSecureHadoop
> has known failure locally)
>  - mvn package -P runSmallTests  -Dsurefire.rerunFailingTestsCount=3
> * A bit of high load testing locally looks good
>
> The compat report has few high sev problems reported but not all need to be
> taken care of given that majority are affecting deprecated/private APIs.
> However, there seems one high sev problem that we might want to consider
> fixing:
>
>
> org/apache/hadoop/hbase/io/encoding/DataBlockEncoding.createEncoder:(Ljava/lang/String;)Lorg/apache/hadoop/hbase/io/encoding/DataBlockEncoder
> Access level has been changed from protected to package-private.
>
> This was done as part of the recent error-prone findings. Either we can
> bring back protected access or, we already have public getter
> *getEncoder()*,
> hence I think perhaps we can make *createEncoder(String
> fullyQualifiedClassName)* as IA.Private. Either way is fine.
>
> Functional testing seems good, thanks Nick.
>
>
> On Wed, Aug 24, 2022 at 2:53 AM Nick Dimiduk  wrote:
>
> > Please vote on this Apache hbase release candidate,
> > hbase-2.5.0RC1
> >
> > The VOTE will remain open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache hbase 2.5.0
> > [ ] -1 Do not release this package because ...
> >
> > The tag to be voted on is 2.5.0RC1:
> >
> >   https://github.com/apache/hbase/tree/2.5.0RC1
> >
> > This tag currently points to git reference
> >
> >   2ecd8bd6d615ca49bfb329b3c0c126c80846d4ab
> >
> > The release files, including signatures, digests, as well as CHANGES.md
> > and RELEASENOTES.md included in this RC can be found at:
> >
> >   https://dist.apache.org/repos/dist/dev/hbase/2.5.0RC1/
> >
> > Maven artifacts are available in a staging repository at:
> >
> >
> https://repository.apache.org/content/repositories/orgapachehbase-1494/
> >
> > Artifacts were signed with the 0x18567F39 key which can be found in:
> >
> >   https://downloads.apache.org/hbase/KEYS
> >
> > To learn more about Apache hbase, please see
> >
> >   http://hbase.apache.org/
> >
> > Thanks,
> > Your HBase Release Manager
> >
>

[jira] [Created] (HBASE-27339) Improve sasl connection failure log message to include server

2022-08-26 Thread Daniel Wong (Jira) 
Daniel Wong created HBASE-27339:
---

 Summary: Improve sasl connection failure log message to include 
server
 Key: HBASE-27339
 URL: https://issues.apache.org/jira/browse/HBASE-27339
 Project: HBase
  Issue Type: Improvement
  Components: Client
Affects Versions: 2.4.13, 1.7.2
Reporter: Daniel Wong
 Fix For: 2.5.0


We were seeing timeout issues in our deployment in the saslConnect path in the 
blockingRPC client.  Knowing which server we failed to connect to would help 
isolate and understand the issue.

 

 
{code:java}
ipc.BlockingRpcConnection - Exception encountered while connecting to the 
server : 
java.io.IOException: Connection reset by peer {code}
 

 

Change the log message to something like

 
{code:java}
ipc.BlockingRpcConnection - Exception encountered while connecting to the 
server myhost.realm.net: 
java.io.IOException: Connection reset by peer {code}
 

See relevant code 
[here|https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/BlockingRpcConnection.java#L398]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] HBase 2.4.14 release candidate (RC1) is available

2022-08-26 Thread Viraj Jasani 
+1

* Signature: ok
* Checksum : ok
* Rat check (1.8.0_341): ok
 - mvn clean apache-rat:check
* Built from source (1.8.0_341): ok
 - mvn clean install  -DskipTests
* Unit tests pass (1.8.0_341): failed (TestUsersOperationsWithSecureHadoop
fails)
 - mvn package -P runSmallTests  -Dsurefire.rerunFailingTestsCount=3
* Manual CRUD testing looks good

Thanks Huaxiang


On Wed, Aug 24, 2022 at 8:59 AM Huaxiang Sun  wrote:

> Please vote on this Apache hbase release candidate,
>
> hbase-2.4.14RC1
>
>
> The VOTE will remain open for at least 72 hours.
>
>
> [ ] +1 Release this package as Apache hbase 2.4.14
>
> [ ] -1 Do not release this package because ...
>
>
> The tag to be voted on is 2.4.14RC1:
>
>
>   https://github.com/apache/hbase/tree/2.4.14RC1
>
>
> This tag currently points to git reference
>
>
>   2e7d75a89271a7479b2f668c4db7a241be3f
>
>
> The release files, including signatures, digests, as well as CHANGES.md
>
> and RELEASENOTES.md included in this RC can be found at:
>
>
>   https://dist.apache.org/repos/dist/dev/hbase/2.4.14RC1/
>
>
> Maven artifacts are available in a staging repository at:
>
>
>   https://repository.apache.org/content/repositories/orgapachehbase-1495/
>
>
> Artifacts were signed with the 0x117C835E key which can be found in:
>
>
>   https://downloads.apache.org/hbase/KEYS
>
>
> To learn more about Apache hbase, please see
>
>
>   http://hbase.apache.org/
>
>
> Thanks,
>
> Your HBase Release Manager
>

Re: [VOTE] Release candidate for HBase 2.5.0 (RC1) is available

2022-08-26 Thread Viraj Jasani 
+0

* Signature: ok
* Checksum : ok
* Rat check (1.8.0_341): ok
 - mvn clean apache-rat:check
* Built from source (1.8.0_341): ok
 - mvn clean install  -DskipTests
* Unit tests pass (1.8.0_341): failed (TestUsersOperationsWithSecureHadoop
has known failure locally)
 - mvn package -P runSmallTests  -Dsurefire.rerunFailingTestsCount=3
* A bit of high load testing locally looks good

The compat report has few high sev problems reported but not all need to be
taken care of given that majority are affecting deprecated/private APIs.
However, there seems one high sev problem that we might want to consider
fixing:

org/apache/hadoop/hbase/io/encoding/DataBlockEncoding.createEncoder:(Ljava/lang/String;)Lorg/apache/hadoop/hbase/io/encoding/DataBlockEncoder
Access level has been changed from protected to package-private.

This was done as part of the recent error-prone findings. Either we can
bring back protected access or, we already have public getter *getEncoder()*,
hence I think perhaps we can make *createEncoder(String
fullyQualifiedClassName)* as IA.Private. Either way is fine.

Functional testing seems good, thanks Nick.


On Wed, Aug 24, 2022 at 2:53 AM Nick Dimiduk  wrote:

> Please vote on this Apache hbase release candidate,
> hbase-2.5.0RC1
>
> The VOTE will remain open for at least 72 hours.
>
> [ ] +1 Release this package as Apache hbase 2.5.0
> [ ] -1 Do not release this package because ...
>
> The tag to be voted on is 2.5.0RC1:
>
>   https://github.com/apache/hbase/tree/2.5.0RC1
>
> This tag currently points to git reference
>
>   2ecd8bd6d615ca49bfb329b3c0c126c80846d4ab
>
> The release files, including signatures, digests, as well as CHANGES.md
> and RELEASENOTES.md included in this RC can be found at:
>
>   https://dist.apache.org/repos/dist/dev/hbase/2.5.0RC1/
>
> Maven artifacts are available in a staging repository at:
>
>   https://repository.apache.org/content/repositories/orgapachehbase-1494/
>
> Artifacts were signed with the 0x18567F39 key which can be found in:
>
>   https://downloads.apache.org/hbase/KEYS
>
> To learn more about Apache hbase, please see
>
>   http://hbase.apache.org/
>
> Thanks,
> Your HBase Release Manager
>

[jira] [Created] (HBASE-27338) brotli compression lib tests fail on arm64

2022-08-26 Thread Michael Stack (Jira) 
Michael Stack created HBASE-27338:
-

 Summary: brotli compression lib tests fail on arm64
 Key: HBASE-27338
 URL: https://issues.apache.org/jira/browse/HBASE-27338
 Project: HBase
  Issue Type: Improvement
Affects Versions: 2.5.0
Reporter: Michael Stack


The brotli tests fail on M1 macs

 

{{[INFO] Running org.apache.hadoop.hbase.io.compress.brotli.TestBrotliCodec}}
{{[INFO] Running 
org.apache.hadoop.hbase.io.compress.brotli.TestHFileCompressionBrotli}}
{{[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.33 
s <<< FAILURE! - in 
org.apache.hadoop.hbase.io.compress.brotli.TestHFileCompressionBrotli}}
{{[ERROR] 
org.apache.hadoop.hbase.io.compress.brotli.TestHFileCompressionBrotli.test  
Time elapsed: 0.225 s  <<< ERROR!}}
{{java.lang.UnsatisfiedLinkError: Failed to load Brotli native library}}

{{...}}

 

The lib is installed on this machine. A new release of 
*[Brotli4j|https://github.com/hyperxpro/Brotli4j]* lib, 1.8.0, done a few days 
ago fixes the issue... (See [https://github.com/hyperxpro/Brotli4j/pull/34).] I 
tried it .

{{[INFO] ---}}
{{[INFO]  T E S T S}}
{{[INFO] ---}}
{{[INFO] Running org.apache.hadoop.hbase.io.compress.brotli.TestBrotliCodec}}
{{[INFO] Running 
org.apache.hadoop.hbase.io.compress.brotli.TestHFileCompressionBrotli}}
{{[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.036 
s - in org.apache.hadoop.hbase.io.compress.brotli.TestHFileCompressionBrotli}}

{{[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.42 s 
- in org.apache.hadoop.hbase.io.compress.brotli.TestBrotliCodec}}
{{[INFO]}}
{{[INFO] Results:}}
{{[INFO]}}
{{[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0}}
{{[INFO]}}
{{[INFO]}}
{{[INFO] --- maven-surefire-plugin:3.0.0-M6:test (secondPartTestsExecution) @ 
hbase-compression-brotli ---}}
{{[INFO] Tests are skipped.}}
{{[INFO]}}
{{[INFO] --- maven-jar-plugin:3.2.0:test-jar (default) @ 
hbase-compression-brotli ---}}
{{[INFO] Building jar: 
/Users/stack/checkouts/hbase/2.5.0RC1/hbase-2.5.0/hbase-compression/hbase-compression-brotli/target/hbase-compression-brotli-2.5.0-tests.jar}}
{{[INFO]}}
{{[INFO] --- maven-jar-plugin:3.2.0:jar (default-jar) @ 
hbase-compression-brotli ---}}
{{[INFO] Building jar: 
/Users/stack/checkouts/hbase/2.5.0RC1/hbase-2.5.0/hbase-compression/hbase-compression-brotli/target/hbase-compression-brotli-2.5.0.jar}}
{{[INFO]}}
{{[INFO] --- maven-site-plugin:3.12.0:attach-descriptor (attach-descriptor) @ 
hbase-compression-brotli ---}}
{{[INFO] Skipping because packaging 'jar' is not pom.}}
{{[INFO]}}
{{[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ 
hbase-compression-brotli ---}}
{{[INFO] Installing 
/Users/stack/checkouts/hbase/2.5.0RC1/hbase-2.5.0/hbase-compression/hbase-compression-brotli/target/hbase-compression-brotli-2.5.0.jar
 to 
/Users/stack/.m2/repository/org/apache/hbase/hbase-compression-brotli/2.5.0/hbase-compression-brotli-2.5.0.jar}}
{{[INFO] Installing 
/Users/stack/checkouts/hbase/2.5.0RC1/hbase-2.5.0/hbase-compression/hbase-compression-brotli/pom.xml
 to 
/Users/stack/.m2/repository/org/apache/hbase/hbase-compression-brotli/2.5.0/hbase-compression-brotli-2.5.0.pom}}
{{[INFO] Installing 
/Users/stack/checkouts/hbase/2.5.0RC1/hbase-2.5.0/hbase-compression/hbase-compression-brotli/target/hbase-compression-brotli-2.5.0-tests.jar
 to 
/Users/stack/.m2/repository/org/apache/hbase/hbase-compression-brotli/2.5.0/hbase-compression-brotli-2.5.0-tests.jar}}
{{[INFO] 
}}
{{[INFO] BUILD SUCCESS}}
{{[INFO] 
}}
{{[INFO] Total time:  16.805 s}}
{{[INFO] Finished at: 2022-08-26T11:30:13-07:00}}
{{[INFO] 
}}

 

 

 

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] Release candidate for HBase 2.5.0 (RC1) is available

2022-08-26 Thread Tak Lon (Stephen) Wu 
+1 (non-binding)

List of items have been checked.

1. normal check with hbase-vote.sh
* Signature: ok
* Checksum : ok
* Rat check (1.8.0_282): ok
 - mvn clean apache-rat:check
* Built from source (1.8.0_282): ok
 - mvn clean install -DskipTests
* Unit tests pass (1.8.0_282): ok
 - mvn package -P runAllTests

2. Release note mostly looks good, few minor comment /
  - [nit] format of HBASE-27055 with HBASE_OPTS seems off.
  - [nit] why does the history of release notes does not
   have section for 2.3.0 and 2.4.0?

3. verified hbase CLI tests, create/put/get/scan/flush/list/disable/drop, ok
4. verified with LTT test, ok
 - bin/hbase ltt -num_keys 100 -write 10:1000:32


[discuss] we can take it out of this thread, but how long
does it take to execute tests with -P runAllTests ?

it took 16:35 hours to have the unit tests completely executed inside docker
runner with 4 CPUs and 10GBI'm wondered how i could have it run faster.

-Stephen


On Fri, Aug 26, 2022 at 8:27 AM 张铎(Duo Zhang)  wrote:
>
> Filed HBASE-27335 and HBASE-27336.
>
> Nick Dimiduk  于2022年8月26日周五 02:10写道：
> >
> > Thanks for checking it out Duo. Please file tickets for both of those
> > issues (if you have not done so yet).
> >
> > -n
> >
> > On Thu, Aug 25, 2022 at 14:43 张铎(Duo Zhang)  wrote:
> >
> > > +1 binding
> > >
> > > Checked sigs and sums: Matched
> > > Rat check: Passed
> > > Built from src: Passed with AdoptOpenJDK-11.0.11+9
> > > Run UTs: Some UTs in hbase-server module failed at the first time and
> > > passed while running the UTs in hbase-server module separately again,
> > > with AdoptOpenJDK-11.0.11+9 and hadoop.profile=3.0
> > > Start a mini cluster: OK with the downloaded binary, the status page
> > > looks almost fine(see issues below)
> > > Run basic (shell) commands: OK with the downloaded client binary.
> > > Loaded 1 rows with LTT, ran 'count' command on the table in shell,
> > > returned 1 rows. Flush command is also fine, and will make the
> > > region visualizer to show something
> > >
> > > Issues:
> > > The region visualizer is interesting, it could show the total region
> > > size, but the region server info just shows as 'undefined'.
> > > When quiting the hbase shell, the command will hang several tens of
> > > seconds before shutting down. Not a blocker as it could finally quit,
> > > but still a bit annoying.
> > >
> > > Thanks Nick!
> > >
> > > Nick Dimiduk  于2022年8月24日周三 17:53写道：
> > > >
> > > > Please vote on this Apache hbase release candidate,
> > > > hbase-2.5.0RC1
> > > >
> > > > The VOTE will remain open for at least 72 hours.
> > > >
> > > > [ ] +1 Release this package as Apache hbase 2.5.0
> > > > [ ] -1 Do not release this package because ...
> > > >
> > > > The tag to be voted on is 2.5.0RC1:
> > > >
> > > >   https://github.com/apache/hbase/tree/2.5.0RC1
> > > >
> > > > This tag currently points to git reference
> > > >
> > > >   2ecd8bd6d615ca49bfb329b3c0c126c80846d4ab
> > > >
> > > > The release files, including signatures, digests, as well as CHANGES.md
> > > > and RELEASENOTES.md included in this RC can be found at:
> > > >
> > > >   https://dist.apache.org/repos/dist/dev/hbase/2.5.0RC1/
> > > >
> > > > Maven artifacts are available in a staging repository at:
> > > >
> > > >
> > > https://repository.apache.org/content/repositories/orgapachehbase-1494/
> > > >
> > > > Artifacts were signed with the 0x18567F39 key which can be found in:
> > > >
> > > >   https://downloads.apache.org/hbase/KEYS
> > > >
> > > > To learn more about Apache hbase, please see
> > > >
> > > >   http://hbase.apache.org/
> > > >
> > > > Thanks,
> > > > Your HBase Release Manager
> > >

[jira] [Created] (HBASE-27337) [hbase-thirdparty] Add nimbus-jose-jwt library to hbase-shaded-miscellaneous

2022-08-26 Thread Andor Molnar (Jira) 
Andor Molnar created HBASE-27337:


 Summary: [hbase-thirdparty] Add nimbus-jose-jwt library to 
hbase-shaded-miscellaneous
 Key: HBASE-27337
 URL: https://issues.apache.org/jira/browse/HBASE-27337
 Project: HBase
  Issue Type: Task
  Components: thirdparty
Reporter: Andor Molnar
Assignee: Andor Molnar


Required for OAuthBearer SASL auth plugin to work properly.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] Release candidate for HBase 2.5.0 (RC1) is available

2022-08-26 Thread Duo Zhang 
Filed HBASE-27335 and HBASE-27336.

Nick Dimiduk  于2022年8月26日周五 02:10写道：
>
> Thanks for checking it out Duo. Please file tickets for both of those
> issues (if you have not done so yet).
>
> -n
>
> On Thu, Aug 25, 2022 at 14:43 张铎(Duo Zhang)  wrote:
>
> > +1 binding
> >
> > Checked sigs and sums: Matched
> > Rat check: Passed
> > Built from src: Passed with AdoptOpenJDK-11.0.11+9
> > Run UTs: Some UTs in hbase-server module failed at the first time and
> > passed while running the UTs in hbase-server module separately again,
> > with AdoptOpenJDK-11.0.11+9 and hadoop.profile=3.0
> > Start a mini cluster: OK with the downloaded binary, the status page
> > looks almost fine(see issues below)
> > Run basic (shell) commands: OK with the downloaded client binary.
> > Loaded 1 rows with LTT, ran 'count' command on the table in shell,
> > returned 1 rows. Flush command is also fine, and will make the
> > region visualizer to show something
> >
> > Issues:
> > The region visualizer is interesting, it could show the total region
> > size, but the region server info just shows as 'undefined'.
> > When quiting the hbase shell, the command will hang several tens of
> > seconds before shutting down. Not a blocker as it could finally quit,
> > but still a bit annoying.
> >
> > Thanks Nick!
> >
> > Nick Dimiduk  于2022年8月24日周三 17:53写道：
> > >
> > > Please vote on this Apache hbase release candidate,
> > > hbase-2.5.0RC1
> > >
> > > The VOTE will remain open for at least 72 hours.
> > >
> > > [ ] +1 Release this package as Apache hbase 2.5.0
> > > [ ] -1 Do not release this package because ...
> > >
> > > The tag to be voted on is 2.5.0RC1:
> > >
> > >   https://github.com/apache/hbase/tree/2.5.0RC1
> > >
> > > This tag currently points to git reference
> > >
> > >   2ecd8bd6d615ca49bfb329b3c0c126c80846d4ab
> > >
> > > The release files, including signatures, digests, as well as CHANGES.md
> > > and RELEASENOTES.md included in this RC can be found at:
> > >
> > >   https://dist.apache.org/repos/dist/dev/hbase/2.5.0RC1/
> > >
> > > Maven artifacts are available in a staging repository at:
> > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehbase-1494/
> > >
> > > Artifacts were signed with the 0x18567F39 key which can be found in:
> > >
> > >   https://downloads.apache.org/hbase/KEYS
> > >
> > > To learn more about Apache hbase, please see
> > >
> > >   http://hbase.apache.org/
> > >
> > > Thanks,
> > > Your HBase Release Manager
> >

Re: [DISCUSS] HBase 2.5 / Hadoop 3 artifacts

2022-08-26 Thread Andrew Purtell 
The security posture of Hadoop 2 in general is a problem, because maintenance 
on that branch is spotty, that is just how it goes. We had the same situation 
with our now EOL branch-1. I know Hadoop released 2.10.2 to address some CVE 
worthy problems but it is unclear if 2.10.2 addresses all known issues, unlike 
3.3.4. Also as you know Hadoop 2 has unpatchable dependencies on org.codehaus 
versions of Jackson and Jetty, which themselves have high scoring CVEs that 
will never be fixed because they are EOL, and other similar issues. Hadoop 3 
doesn’t completely solve such problems but is the only realistic place we can 
hope they can be addressed as required. For organizations that implement or 
require a top to bottom security audit of their software bill of materials, it 
seems possible to avoid user pain by providing supported convenience artifacts 
*and* libraries built against Hadoop 3 APIs in the Apache repository 
addressable with a Maven classifier. 

My employer has some interests in this area that align so I would like to 
sponsor (implement, review, commit, RM backfill releases, etc.) this work. 
Would there be any objections? Read through the thread for some thoughts on 
approach. Summarized:

- Amend create-release to build, stage, and deploy a -hadoop3 variant build by 
activating the Hadoop 3 build profile. 

- Amend the Hadoop 3 build profile to flatten POMs before deployment to resolve 
potential downstream issues due to Hadoop 3 being a non-default build profile. 
(This could also be applied to all builds.)

- Amend hbase-vote to be aware of and evaluate if present -hadoop3 variant 
artifacts. 


> On Aug 25, 2022, at 10:40 AM, Andrew Purtell  wrote:
> 
> Thanks, that would work. 
> 
>> On Aug 25, 2022, at 11:35 AM, Sean Busbey  wrote:
>> 
>> yes, the flatten plugin. We use it in hbase-connectors already.
>> 
>> https://www.mojohaus.org/flatten-maven-plugin/
>> 
>> this sounds like it could also be a use case for BOMs, which would also
>> benefit users of our client artifacts that use build tools that don't
>> respect maven profiles generally, like gradle.
>> 
>>> On Thu, Aug 25, 2022 at 10:30 AM Andrew Purtell 
>>> wrote:
>>> 
>>> Thinking about this a bit more, we will have an issue in that the POMs
>>> published from our -hadoop3 build will not have a default activation of our
>>> Hadoop 3 build profile. The convenience binaries will function as expected
>>> but Maven will read and process eg Phoenix POMs, then download and perform
>>> substitutions on HBase POMs, and then etc, so downstreamers like Phoenix
>>> will have to set up the hadoop.profile variable for us in their default
>>> build profile or else the transitive paths through us may be wrong. I
>>> wonder if there is a Maven plugin available for deploying POMs with all
>>> variable substitutions performed before deployment, that would solve that
>>> problem and all conceivable related issues.
>>> 
 On Aug 25, 2022, at 11:03 AM, Andrew Purtell 
>>> wrote:
 
 I think 2.x is going to have a few years of life remaining so it would
>>> be best, if we are going to address this, to have a 2.x solution was well
>>> as a 3.x one.
 
 In my opinion we can continue to publish 2.4 and 2.5 (and 2.6) unchanged
>>> and then also introduce a Hadoop 3 release using “hadoop3” or similar as
>>> Maven classifier. Phoenix could specify this classifier in their POMs.
>>> Everyone should be happy. Users who already are comfortable with the Hadoop
>>> 2 default don’t have to change anything. A one time POM change on the
>>> Phoenix side is required but that’s it.
 
 The additional build time complexity for generating two releases can be
>>> incorporated into create-release. Nobody does manual releases any more as
>>> far as I know. Likewise, download and verification of -hadoop3 convenience
>>> binaries can be added to hbase-vote. I believe we are all using that tool
>>> for verification of releases now. After these one time changes are landed
>>> the cost for RMs and PMC will be only in a roughly doubled amount of time
>>> needed to build and verify releases.
 
>> On Aug 17, 2022, at 9:06 AM, Nick Dimiduk  wrote:
>> 
>> Hi Geoffrey,
>> 
>> I have no complaints with shipping convenience binaries built against
>>> both
> Hadoop2 and Hadoop3. The primary challenge is implementing the
> necessary build changes, the secondary challenge is verifying/testing it
> works reliably.
> 
> But for Phoenix, are you asking for convenience binaries, or are you
>>> asking
> for artifacts published into maven that have the Hadoop3 profile
>>> activated
> and specify the associated dependencies?
> 
> I'm afraid that the 2.5.0 release ship has already sailed. I've heard
>>> talk
> of a 2.6 "fast-follow", so maybe someone can have the build changes
>>> ready
> for that? Also, isn't this a too little, too late situation? Shouldn't
>>> we
> shift our focus to releasing

[jira] [Created] (HBASE-27336) The region visualizer shows 'undefined' region server

2022-08-26 Thread Duo Zhang (Jira) 
Duo Zhang created HBASE-27336:
-

 Summary: The region visualizer shows 'undefined' region server
 Key: HBASE-27336
 URL: https://issues.apache.org/jira/browse/HBASE-27336
 Project: HBase
  Issue Type: Bug
  Components: master, UI
Reporter: Duo Zhang






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27335) HBase shell hang fora minute when quiting

2022-08-26 Thread Duo Zhang (Jira) 
Duo Zhang created HBASE-27335:
-

 Summary: HBase shell hang fora minute when quiting
 Key: HBASE-27335
 URL: https://issues.apache.org/jira/browse/HBASE-27335
 Project: HBase
  Issue Type: Bug
  Components: shell
Reporter: Duo Zhang


Found this when trying the 2.5.0 RC. The way to reproduce

Use './bin/hbase shell' to connect to a cluster, and then run these two commands

{code}
count "hbase:meta"
quit
{code}

We will hang about 1 minute before the final exit of the shell process.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (HBASE-25922) Disabled sanity checks ignored on snapshot restore

2022-08-26 Thread Bryan Beaudreault (Jira) 


 [ 
https://issues.apache.org/jira/browse/HBASE-25922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Beaudreault resolved HBASE-25922.
---
Fix Version/s: 2.6.0
   2.5.1
   3.0.0-alpha-4
   Resolution: Fixed

Pushed to master, branch-2, branch-2.5. Thanks for the contribution [~ukumar] 
and review [~zhangduo]! 

> Disabled sanity checks ignored on snapshot restore
> --
>
> Key: HBASE-25922
> URL: https://issues.apache.org/jira/browse/HBASE-25922
> Project: HBase
>  Issue Type: Bug
>  Components: conf, snapshots
>Affects Versions: 2.2.6, 2.4.2
> Environment: This has been tested in
>  * Google Dataproc running HBase 2.2.6
>  * Local HBase 2.4.2
>Reporter: Julian Nodorp
>Assignee: Ujjawal Kumar
>Priority: Minor
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4
>
>
> Disabling sanity checks on a table is ignored when restoring snapshots. If 
> this is expected behavior at least the error message is misleading.
> h3. Steps *to Reproduce*
>  # Create a new table
> {{create 't', 'cf'}}
>  # Add a coprocessor to the newly created table
> {{alter 't', METHOD => 'table_att', 'coprocessor' => 
> 'coprocessor.jar|com.example.MyCoprocessor|0'}}
>  # Create a snapshot
> {{snapshot 't', 'snapshot-t'}}
>  # Disable the table to prevent region servers from crashing in the next step
> {{disable 't'}}
>  # Delete the coprocessor JAR and restart HBase.
>  # Attempt to restore the snapshot leads to failing sanity check as expected
> {{restore_snapshot 'snapshot-t'}}
> {{ERROR: org.apache.hadoop.hbase.DoNotRetryIOException: coprocessor.jar Set 
> hbase.table.sanity.checks to false at conf or table descriptor if you want to 
> bypass sanity checks [...]}}
>  # Disable sanity checks (as described in the error message) and retry
> {{alter 't', CONFIGURATION => \{'hbase.table.sanity.checks' => 'false'}}}
> {{restore_snapshot 'snapshot-t'}}
> h3. Expected Behavior
> The snapshot is restored.
> h3. Actual Behavior
> The same error message as in step 6. is shown.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27334) Add support for unsigned varint to Bytes util class

2022-08-26 Thread Bryan Beaudreault (Jira) 
Bryan Beaudreault created HBASE-27334:
-

 Summary: Add support for unsigned varint to Bytes util class
 Key: HBASE-27334
 URL: https://issues.apache.org/jira/browse/HBASE-27334
 Project: HBase
  Issue Type: New Feature
Reporter: Bryan Beaudreault


Our current Bytes.vintToBytes accepts a signed long, and the resulting encoded 
size is larger than it needs to be for small values. For example, 
Bytes.vintToBytes requires 3 bytes for values over 255, whereas an unsigned 
implementation can use 2 bytes for up to 16383. This continues across the 
spectrum, with the current requiring 4 bytes for values over 65535 and an 
unsigned impl can use 3 bytes up to 2097151, etc.

We can pull the implementation from CodedInputStream and CodedOutputStream



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27333) The condition for using the local hostname is the opposite

2022-08-26 Thread Xiaolin Ha (Jira) 
Xiaolin Ha created HBASE-27333:
--

 Summary: The condition for using the local hostname is the opposite
 Key: HBASE-27333
 URL: https://issues.apache.org/jira/browse/HBASE-27333
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 2.4.13, 3.0.0-alpha-3
Reporter: Xiaolin Ha
Assignee: Xiaolin Ha
 Fix For: 2.5.1, 3.0.0-alpha-4


For HRegionServer#handleReportForDutyResponse, when the hostname is different 
from the regionserver and master side, the condition to use the local hostname 
configed on regionserver is opposite.
{code:java}
if (
  !StringUtils.isBlank(useThisHostnameInstead)
&& !hostnameFromMasterPOV.equals(useThisHostnameInstead)
) {
  String msg = "Master passed us a different hostname to use; was="
+ this.useThisHostnameInstead + ", but now=" + hostnameFromMasterPOV;
  LOG.error(msg);
  throw new IOException(msg);
}
if (
  StringUtils.isBlank(useThisHostnameInstead)
&& 
!hostnameFromMasterPOV.equals(rpcServices.getSocketAddress().getHostName())
) {
  String msg = "Master passed us a different hostname to use; was="
+ rpcServices.getSocketAddress().getHostName() + ", but now=" + 
hostnameFromMasterPOV;
  LOG.error(msg);
} {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] HBase 2.4.14 release candidate (RC1) is available

2022-08-26 Thread Peter Somogyi 
+1 (binding)

* Signature: ok
* Checksum : ok
* Rat check (1.8.0_302): ok
 - mvn clean apache-rat:check
* Built from source (1.8.0_302): ok
 - mvn clean install  -DskipTests
* Unit tests pass (1.8.0_302): ok
 - mvn package -P runAllTests  -Dsurefire.rerunFailingTestsCount=3
* Compatibility report: ok
* Changes, Release notes: minor issue
 - HBASE-27089 title contains “ characters which result in a
markdown display issue
 Add “commons.crypto.stream.buffer.size†configuration

Thanks,
Peter

On Thu, Aug 25, 2022 at 2:27 PM Nick Dimiduk  wrote:

> +1
>
> * Compatibility Report: ok
> * Jenkins CI history: ok
> * Signature: ok
> * Checksum : ok
> * Rat check (1.8.0_292): ok
>  - mvn clean apache-rat:check -D skipTests
> * Rat check (11.0.11): ok
>  - mvn clean apache-rat:check -D skipTests -D hadoop.profile=3.0
> * Built from source (1.8.0_292): ok
>  - mvn clean install -D skipTests -DskipTests
> * Built from source (11.0.11): ok
>  - mvn clean install -D skipTests -D hadoop.profile=3.0 -DskipTests
>
>
> On Wed, Aug 24, 2022 at 5:59 PM Huaxiang Sun 
> wrote:
>
> > Please vote on this Apache hbase release candidate,
> >
> > hbase-2.4.14RC1
> >
> >
> > The VOTE will remain open for at least 72 hours.
> >
> >
> > [ ] +1 Release this package as Apache hbase 2.4.14
> >
> > [ ] -1 Do not release this package because ...
> >
> >
> > The tag to be voted on is 2.4.14RC1:
> >
> >
> >   https://github.com/apache/hbase/tree/2.4.14RC1
> >
> >
> > This tag currently points to git reference
> >
> >
> >   2e7d75a89271a7479b2f668c4db7a241be3f
> >
> >
> > The release files, including signatures, digests, as well as CHANGES.md
> >
> > and RELEASENOTES.md included in this RC can be found at:
> >
> >
> >   https://dist.apache.org/repos/dist/dev/hbase/2.4.14RC1/
> >
> >
> > Maven artifacts are available in a staging repository at:
> >
> >
> >
> https://repository.apache.org/content/repositories/orgapachehbase-1495/
> >
> >
> > Artifacts were signed with the 0x117C835E key which can be found in:
> >
> >
> >   https://downloads.apache.org/hbase/KEYS
> >
> >
> > To learn more about Apache hbase, please see
> >
> >
> >   http://hbase.apache.org/
> >
> >
> > Thanks,
> >
> > Your HBase Release Manager
> >
>

[jira] [Created] (HBASE-27332) Remove RejectedExecutionHandler for long/short compaction thread pool

2022-08-26 Thread Xiaolin Ha (Jira) 
Xiaolin Ha created HBASE-27332:
--

 Summary: Remove RejectedExecutionHandler for long/short compaction 
thread pool
 Key: HBASE-27332
 URL: https://issues.apache.org/jira/browse/HBASE-27332
 Project: HBase
  Issue Type: Improvement
  Components: Compaction
Affects Versions: 2.4.13, 3.0.0-alpha-3
Reporter: Xiaolin Ha
 Fix For: 2.6.0, 3.0.0-alpha-4


As disscussed in 
[https://github.com/apache/hbase/pull/4725|https://github.com/apache/hbase/pull/4725,],

actually the max size of StealJobQueue is bouded by the VM limit of an array, 
and the OOM exception occurs before the rejection handler. So StealJobQueue is 
equivalent to be unbounded. I think the RejectionHandler may bring some 
confusions and make the codes a little puzzle.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)