svn commit: r41309 [3/4] - /dev/hbase/2.2.6RC2/
Added: dev/hbase/2.2.6RC2/RELEASENOTES.md
==
--- dev/hbase/2.2.6RC2/RELEASENOTES.md (added)
+++ dev/hbase/2.2.6RC2/RELEASENOTES.md Fri Sep 4 02:46:04 2020
@@ -0,0 +1,2112 @@
+# RELEASENOTES
+
+
+# HBASE 2.2.6 Release Notes
+
+These release notes cover new developer and user-facing incompatibilities,
important issues, features, and major improvements.
+
+
+---
+
+* [HBASE-24892](https://issues.apache.org/jira/browse/HBASE-24892) | *Major* |
**config 'hbase.hregion.memstore.mslab.indexchunksize' not be used**
+
+Remove the config "hbase.hregion.memstore.mslab.indexchunksize" which never
used. And use "hbase.hregion.memstore.mslab.indexchunksize.percent" instead.
+
+
+---
+
+* [HBASE-24150](https://issues.apache.org/jira/browse/HBASE-24150) | *Major* |
**Allow module tests run in parallel**
+
+Pass -T2 to mvn. Makes it so we do two modules-at-a-time dependencies willing.
Helps speed build and testing. Doubles the resource usage when running modules
in parallel.
+
+
+---
+
+* [HBASE-24126](https://issues.apache.org/jira/browse/HBASE-24126) | *Major* |
**Up the container nproc uplimit from 1 to 12500**
+
+Start docker with upped ulimit for nproc passing '--ulimit nproc=12500'. It
was 1, the default, but made it 12500. Then, set PROC\_LIMIT in
hbase-personality so when yetus runs, it is w/ the new 12500 value.
+
+
+---
+
+* [HBASE-24625](https://issues.apache.org/jira/browse/HBASE-24625) |
*Critical* | **AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the
expected synced file length.**
+
+We add a method getSyncedLength in WALProvider.WriterBase interface for
WALFileLengthProvider used for replication, considering the case if we use
AsyncFSWAL,we write to 3 DNs concurrently,according to the visibility guarantee
of HDFS, the data will be available immediately
+when arriving at DN since all the DNs will be considered as the last one in
pipeline.This means replication may read uncommitted data and replicate it to
the remote cluster and cause data inconsistency.The method WriterBase#getLength
may return length which just in hdfs client buffer and not successfully synced
to HDFS, so we use this method WriterBase#getSyncedLength to return the length
successfully synced to HDFS and replication thread could only read writing WAL
file limited by this length.
+see also HBASE-14004 and this document for more details:
+https://docs.google.com/document/d/11AyWtGhItQs6vsLRIx32PwTxmBY3libXwGXI25obVEY/edit#
+
+Before this patch, replication may read uncommitted data and replicate it to
the slave cluster and cause data inconsistency between master and slave
cluster, we could use FSHLog instead of AsyncFSWAL to reduce probability of
inconsistency without this patch applied.
+
+
+---
+
+* [HBASE-24578](https://issues.apache.org/jira/browse/HBASE-24578) | *Major* |
**[WAL] Add a parameter to config RingBufferEventHandler's SyncFuture count**
+
+Introduce a new parameter "hbase.regionserver.wal.sync.batch.count" to control
the wal sync batch size which is equals to "hbase.regionserver.handler.count"
by default. It should work well if you use default wal provider---one wal per
regionserver. But if you use read/write separated handlers, you can set
"hbase.regionserver.wal.sync.batch.count" to the number of write handlers. And
if you use wal-per-groups or wal-per-region, you can consider lower
"hbase.regionserver.wal.sync.batch.count", the default number will be too big
and consume more memories as the number of wals grows.
+
+
+---
+
+* [HBASE-24603](https://issues.apache.org/jira/browse/HBASE-24603) |
*Critical* | **Zookeeper sync() call is async**
+
+
+
+Fixes a couple of bugs in ZooKeeper interaction. Firstly, zk sync() call that
is used to sync the lagging followers with leader so that the client sees a
consistent snapshot state was actually asynchronous under the hood. We make it
synchronous for correctness. Second, zookeeper events are now processed in a
separate thread rather than doing it in the thread context of zookeeper client
connection. This decoupling frees up client connection quickly and avoids
deadlocks.
+
+
+---
+
+* [HBASE-24205](https://issues.apache.org/jira/browse/HBASE-24205) | *Major* |
**Create metric to know the number of reads that happens from memstore**
+
+Adds a new metric where we collect the number of read requests (tracked per
row) whether the row was fetched completely from memstore or it was pulled from
files and memstore.
+The metric is now collected under the mbean for Tables and under the mbean for
regions.
+Under table mbean ie.-
+'name": "Hadoop:service=HBase,name=RegionServer,sub=Tables'
+The new metrics will be listed as
+{code}
+
"Namespace\_default\_table\_t3\_columnfamily\_f1\_metric\_memstoreOnlyRowReadsCount":
5,
+
"Namespace\_default\_table\_t3\_columnfamily\_f1\_metric\_mixedRowReadsCount":
1,
+{code}
+Where the format is
svn commit: r41309 [1/4] - /dev/hbase/2.2.6RC2/
Author: zghao Date: Fri Sep 4 02:46:04 2020 New Revision: 41309 Log: Apache HBase 2.2.6RC2 Added: dev/hbase/2.2.6RC2/ dev/hbase/2.2.6RC2/CHANGES.md dev/hbase/2.2.6RC2/RELEASENOTES.md dev/hbase/2.2.6RC2/api_compare_2.2.6RC2_to_2.2.5.html dev/hbase/2.2.6RC2/hbase-2.2.6-bin.tar.gz (with props) dev/hbase/2.2.6RC2/hbase-2.2.6-bin.tar.gz.asc dev/hbase/2.2.6RC2/hbase-2.2.6-bin.tar.gz.sha512 dev/hbase/2.2.6RC2/hbase-2.2.6-client-bin.tar.gz (with props) dev/hbase/2.2.6RC2/hbase-2.2.6-client-bin.tar.gz.asc dev/hbase/2.2.6RC2/hbase-2.2.6-client-bin.tar.gz.sha512 dev/hbase/2.2.6RC2/hbase-2.2.6-src.tar.gz (with props) dev/hbase/2.2.6RC2/hbase-2.2.6-src.tar.gz.asc dev/hbase/2.2.6RC2/hbase-2.2.6-src.tar.gz.sha512
svn commit: r41309 [2/4] - /dev/hbase/2.2.6RC2/
Added: dev/hbase/2.2.6RC2/CHANGES.md == --- dev/hbase/2.2.6RC2/CHANGES.md (added) +++ dev/hbase/2.2.6RC2/CHANGES.md Fri Sep 4 02:46:04 2020 @@ -0,0 +1,1542 @@ +# HBASE Changelog + + +## Release 2.2.6 - Unreleased (as of 2020-09-04) + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-24913](https://issues.apache.org/jira/browse/HBASE-24913) | Refactor TestJMXConnectorServer | Major | test | +| [HBASE-24898](https://issues.apache.org/jira/browse/HBASE-24898) | Use EnvironmentEdge.currentTime() instead of System.currentTimeMillis() in CurrentHourProvider | Major | tooling | +| [HBASE-24928](https://issues.apache.org/jira/browse/HBASE-24928) | balanceRSGroup should skip generating balance plan for disabled table and splitParent region | Major | Balancer | +| [HBASE-24686](https://issues.apache.org/jira/browse/HBASE-24686) | [LOG] Log improvement in Connection#close | Major | Client, logging | +| [HBASE-24912](https://issues.apache.org/jira/browse/HBASE-24912) | Enlarge MemstoreFlusherChore/CompactionChecker period for unit test | Major | . | +| [HBASE-24854](https://issues.apache.org/jira/browse/HBASE-24854) | Correct the help content of assign and unassign commands in hbase shell | Minor | shell | +| [HBASE-21721](https://issues.apache.org/jira/browse/HBASE-21721) | FSHLog : reduce write#syncs() times | Major | . | +| [HBASE-24821](https://issues.apache.org/jira/browse/HBASE-24821) | Simplify the logic of getRegionInfo in TestFlushFromClient to reduce redundancy code | Minor | test | +| [HBASE-20226](https://issues.apache.org/jira/browse/HBASE-20226) | Performance Improvement Taking Large Snapshots In Remote Filesystems | Minor | snapshots | +| [HBASE-24757](https://issues.apache.org/jira/browse/HBASE-24757) | ReplicationSink should limit the batch rowcount for batch mutations based on hbase.rpc.rows.warning.threshold | Major | . | +| [HBASE-24777](https://issues.apache.org/jira/browse/HBASE-24777) | InfoServer support ipv6 host and port | Minor | UI | +| [HBASE-24696](https://issues.apache.org/jira/browse/HBASE-24696) | Include JVM information on Web UI under "Software Attributes" | Minor | UI | +| [HBASE-24740](https://issues.apache.org/jira/browse/HBASE-24740) | Enable journal logging for HBase snapshot operation | Minor | master | +| [HBASE-24578](https://issues.apache.org/jira/browse/HBASE-24578) | [WAL] Add a parameter to config RingBufferEventHandler's SyncFuture count | Major | wal | +| [HBASE-24467](https://issues.apache.org/jira/browse/HBASE-24467) | Backport HBASE-23963: Split TestFromClientSide; it takes too long to complete timing out | Major | . | +| [HBASE-24562](https://issues.apache.org/jira/browse/HBASE-24562) | Stabilize master startup with meta replicas enabled | Major | meta, read replicas | +| [HBASE-24603](https://issues.apache.org/jira/browse/HBASE-24603) | Zookeeper sync() call is async | Critical | master, regionserver | +| [HBASE-24205](https://issues.apache.org/jira/browse/HBASE-24205) | Create metric to know the number of reads that happens from memstore | Major | metrics | +| [HBASE-24605](https://issues.apache.org/jira/browse/HBASE-24605) | Break long region names in the web UI | Minor | UI | +| [HBASE-24102](https://issues.apache.org/jira/browse/HBASE-24102) | RegionMover should exclude draining/decommissioning nodes from target RSs | Major | . | +| [HBASE-24592](https://issues.apache.org/jira/browse/HBASE-24592) | [brnach-2.2] Upgrade junit to 4.13 | Major | . | +| [HBASE-24478](https://issues.apache.org/jira/browse/HBASE-24478) | The regionInfo parameter for MasterProcedureScheduler#waitRegions and MasterProcedureScheduler#wakeRegions should be plural | Minor | proc-v2 | +| [HBASE-24524](https://issues.apache.org/jira/browse/HBASE-24524) | SyncTable logging improvements | Minor | . | +| [HBASE-24483](https://issues.apache.org/jira/browse/HBASE-24483) | Add repeated prefix logging for MultipleColumnPrefixFilter | Minor | Filters | +| [HBASE-24504](https://issues.apache.org/jira/browse/HBASE-24504) | refactor call setupCluster/tearDownCluster in TestTableSnapshotInputFormat | Minor | mapreduce, test | +| [HBASE-24412](https://issues.apache.org/jira/browse/HBASE-24412) | Canary support check only one column family per RegionTask | Major | canary | +| [HBASE-24455](https://issues.apache.org/jira/browse/HBASE-24455) | Correct the doc of "On the number of column families" | Minor | documentation | +| [HBASE-24470](https://issues.apache.org/jira/browse/HBASE-24470) | Add store file info when log meessages in HStore. | Minor | logging, regionserver | +| [HBASE-24416](https://issues.apache.org/jira/browse/HBASE-24416) | RegionNormalizer spliting region should not be limited by hbase.normalizer.min.region.count | Major | . | +|
svn commit: r41309 [4/4] - /dev/hbase/2.2.6RC2/
Added: dev/hbase/2.2.6RC2/api_compare_2.2.6RC2_to_2.2.5.html
==
--- dev/hbase/2.2.6RC2/api_compare_2.2.6RC2_to_2.2.5.html (added)
+++ dev/hbase/2.2.6RC2/api_compare_2.2.6RC2_to_2.2.5.html Fri Sep 4 02:46:04
2020
@@ -0,0 +1,619 @@
+
+
+http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd;>
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+
+
+
+
+
+hbase: rel/2.2.5 to 2.2.6RC2 compatibility report
+
+body {
+font-family:Arial, sans-serif;
+background-color:White;
+color:Black;
+}
+hr {
+color:Black;
+background-color:Black;
+height:1px;
+border:0;
+}
+h1 {
+margin-bottom:0px;
+padding-bottom:0px;
+font-size:1.625em;
+}
+h2 {
+margin-bottom:0px;
+padding-bottom:0px;
+font-size:1.25em;
+white-space:nowrap;
+}
+div.symbols {
+color:#003E69;
+}
+div.symbols i {
+color:Brown;
+}
+span.section {
+font-weight:bold;
+cursor:pointer;
+color:#003E69;
+white-space:nowrap;
+margin-left:0.3125em;
+}
+span:hover.section {
+color:#336699;
+}
+span.sect_aff {
+cursor:pointer;
+padding-left:1.55em;
+font-size:0.875em;
+color:#cc3300;
+}
+span.ext {
+font-weight:normal;
+}
+span.jar {
+color:#cc3300;
+font-size:0.875em;
+font-weight:bold;
+}
+div.jar_list {
+padding-left:0.4em;
+font-size:0.94em;
+}
+span.pkg_t {
+color:#408080;
+font-size:0.875em;
+}
+span.pkg {
+color:#408080;
+font-size:0.875em;
+font-weight:bold;
+}
+span.cname {
+color:Green;
+font-size:0.875em;
+font-weight:bold;
+}
+span.iname_b {
+font-weight:bold;
+}
+span.iname_a {
+color:#33;
+font-weight:bold;
+font-size:0.94em;
+}
+span.sym_p {
+font-weight:normal;
+white-space:normal;
+}
+span.sym_pd {
+white-space:normal;
+}
+span.sym_p span, span.sym_pd span {
+white-space:nowrap;
+}
+span.attr {
+color:Black;
+font-weight:normal;
+}
+span.deprecated {
+color:Red;
+font-weight:bold;
+font-family:Monaco, monospace;
+}
+div.affect {
+padding-left:1em;
+padding-bottom:10px;
+font-size:0.87em;
+font-style:italic;
+line-height:0.9em;
+}
+div.affected {
+padding-left:2em;
+padding-top:10px;
+}
+table.ptable {
+border-collapse:collapse;
+border:1px outset black;
+margin-left:0.95em;
+margin-top:3px;
+margin-bottom:3px;
+width:56.25em;
+}
+table.ptable td {
+border:1px solid Gray;
+padding:3px;
+font-size:0.875em;
+text-align:left;
+vertical-align:top;
+max-width:28em;
+word-wrap:break-word;
+}
+table.ptable th {
+background-color:#ee;
+font-weight:bold;
+color:#33;
+font-family:Verdana, Arial;
+font-size:0.875em;
+border:1px solid Gray;
+text-align:center;
+vertical-align:top;
+white-space:nowrap;
+padding:3px;
+}
+table.summary {
+border-collapse:collapse;
+border:1px outset black;
+}
+table.summary th {
+background-color:#ee;
+font-weight:normal;
+text-align:left;
+font-size:0.94em;
+white-space:nowrap;
+border:1px inset Gray;
+padding:3px;
+}
+table.summary td {
+text-align:right;
+white-space:nowrap;
+border:1px inset Gray;
+padding:3px 5px 3px 10px;
+}
+span.mngl {
+padding-left:1em;
+font-size:0.875em;
+cursor:text;
+color:#44;
+font-weight:bold;
+}
+span.pleft {
+padding-left:2.5em;
+}
+span.color_p {
+font-style:italic;
+color:Brown;
+}
+span.param {
+font-style:italic;
+}
+span.focus_p {
+font-style:italic;
+background-color:#DCDCDC;
+}
+span.ttype {
+font-weight:normal;
+}
+span.nowrap {
+white-space:nowrap;
+}
+span.value {
+white-space:nowrap;
+font-weight:bold;
+}
+.passed {
+background-color:#CCFFCC;
+font-weight:normal;
+}
+.warning {
+background-color:#F4F4AF;
+font-weight:normal;
+}
+.failed {
+background-color:#FF;
+font-weight:normal;
+}
+.new {
+background-color:#C6DEFF;
+font-weight:normal;
+}
+
+.compatible {
+background-color:#CCFFCC;
+font-weight:normal;
+}
+.almost_compatible {
+background-color:#FFDAA3;
+font-weight:normal;
+}
+.incompatible {
+background-color:#FF;
+font-weight:normal;
+}
+.gray {
+background-color:#DCDCDC;
+font-weight:normal;
+}
+
+.top_ref {
+font-size:0.69em;
+}
+.footer {
+font-size:0.8125em;
+}
+.tabset {
+float:left;
+}
+a.tab {
+border:1px solid Black;
+float:left;
+margin:0px 5px -1px 0px;
+padding:3px 5px 3px 5px;
+position:relative;
+font-size:0.875em;
+background-color:#DDD;
+text-decoration:none;
+color:Black;
+}
+a.disabled:hover
+{
+color:Black;
+background:#EEE;
+}
+a.active:hover
+{
+color:Black;
+background:White;
+}
+a.active {
+border-bottom-color:White;
+background-color:White;
+}
+div.tab {
+border-top:1px solid
[hbase] annotated tag 2.2.6RC2 created (now a44cf62)
This is an automated email from the ASF dual-hosted git repository. zghao pushed a change to annotated tag 2.2.6RC2 in repository https://gitbox.apache.org/repos/asf/hbase.git. at a44cf62 (tag) tagging 88c9a386176e2c2b5fd9915d0e9d3ce17d0e456e (commit) replaces 2.2.6RC1 by Guanghao Zhang on Fri Sep 4 09:44:20 2020 +0800 - Log - Tagging the 2.2.6 third Release Candidate -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQIcBAABAgAGBQJfUZv0AAoJEAKVRF0X8TRurPcP/39bAqL6rjafzmz1slvY5gmd h2zfUlRFn0pxMg/2EudFCqo6gt5L65TyXFUvsH707pxStXSA35B6kcgYEwO2J9b2 0buSNCnFcTBWHZM5BjFdlpxkBnYOBJDh/bVNnxf9yklnQT3fE06sgcDNLqHkLqLO F+4VJj/64YtaRjvbwjI4cubv/X4VSTKoaP+t6HREFSALYX2qWsGcgGRkeYmejdOZ 8xwFwzOy/e0+jPbA/+h0QNzKI8XXwuiqGKT3m5pKe7Y/TiiE6YJhtDF0nLnEM6+7 LJpE0aOhB0Qjs+M0Q1Yhd1J9LFnNVzMJt6OXcMvt8rwu0a+IEZKxn3YEjAy2WEDW EbnF6aFFzjVseQny2vygiHIR0I2Hqxfn0LA7mdJ7iFsZ/bqiJoE4dEnQssFD6NNG fb4nH7uKIu8jE7wEHSyLMgaZgeuXbDsCCV7wa23QWZfMTz/mt+MOSiABu7LKXkE1 Kw+Q1A2EhyIuMgurxgVtImRXmE6HJLzXgDd8hVsbK9C/cE8eLFdzLaPq60Xbzt0U z+xlGTbhyAJ3zqcElErt0il0E3RIbgoBAtNIn4q6zEsntm5P+77183qI0buAVStn InRuQlHVFj7/guV+tXXTBpXlaeptlTDYuHCn7jc6vx6uFuk2dQnaVA01xBN535et Tq0iapccPGVkhtBsbjr0 =uEg/ -END PGP SIGNATURE- --- No new revisions were added by this update.
[hbase] branch branch-2.2 updated: HBASE-24689 Generate CHANGES.md and RELEASENOTES.md for 2.2.6 (addendum)
This is an automated email from the ASF dual-hosted git repository. zghao pushed a commit to branch branch-2.2 in repository https://gitbox.apache.org/repos/asf/hbase.git The following commit(s) were added to refs/heads/branch-2.2 by this push: new 88c9a38 HBASE-24689 Generate CHANGES.md and RELEASENOTES.md for 2.2.6 (addendum) 88c9a38 is described below commit 88c9a386176e2c2b5fd9915d0e9d3ce17d0e456e Author: Guanghao Zhang AuthorDate: Fri Sep 4 09:32:31 2020 +0800 HBASE-24689 Generate CHANGES.md and RELEASENOTES.md for 2.2.6 (addendum) --- CHANGES.md | 10 -- RELEASENOTES.md | 8 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index 15f25c1..e754cc1 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -20,13 +20,15 @@ # Be careful doing manual edits in this file. Do not change format # of release header or remove the below marker. This file is generated. # DO NOT REMOVE THIS MARKER; FOR INTERPOLATING CHANGES!--> -## Release 2.2.6 - Unreleased (as of 2020-08-26) +## Release 2.2.6 - Unreleased (as of 2020-09-04) ### IMPROVEMENTS: | JIRA | Summary | Priority | Component | |: |: | :--- |: | +| [HBASE-24913](https://issues.apache.org/jira/browse/HBASE-24913) | Refactor TestJMXConnectorServer | Major | test | +| [HBASE-24898](https://issues.apache.org/jira/browse/HBASE-24898) | Use EnvironmentEdge.currentTime() instead of System.currentTimeMillis() in CurrentHourProvider | Major | tooling | | [HBASE-24928](https://issues.apache.org/jira/browse/HBASE-24928) | balanceRSGroup should skip generating balance plan for disabled table and splitParent region | Major | Balancer | | [HBASE-24686](https://issues.apache.org/jira/browse/HBASE-24686) | [LOG] Log improvement in Connection#close | Major | Client, logging | | [HBASE-24912](https://issues.apache.org/jira/browse/HBASE-24912) | Enlarge MemstoreFlusherChore/CompactionChecker period for unit test | Major | . | @@ -64,6 +66,10 @@ | JIRA | Summary | Priority | Component | |: |: | :--- |: | +| [HBASE-24656](https://issues.apache.org/jira/browse/HBASE-24656) | [Flakey Tests] branch-2 TestMasterNoCluster.testStopDuringStart | Major | . | +| [HBASE-19352](https://issues.apache.org/jira/browse/HBASE-19352) | Port HADOOP-10379: Protect authentication cookies with the HttpOnly and Secure flags | Major | . | +| [HBASE-24971](https://issues.apache.org/jira/browse/HBASE-24971) | Upgrade JQuery to 3.5.1 | Major | security, UI | +| [HBASE-24892](https://issues.apache.org/jira/browse/HBASE-24892) | config 'hbase.hregion.memstore.mslab.indexchunksize' not be used | Major | . | | [HBASE-24897](https://issues.apache.org/jira/browse/HBASE-24897) | RegionReplicaFlushHandler should handle NoServerForRegionException to avoid aborting RegionServer | Major | . | | [HBASE-23987](https://issues.apache.org/jira/browse/HBASE-23987) | NettyRpcClientConfigHelper will not share event loop by default which is incorrect | Major | Client, rpc | | [HBASE-24871](https://issues.apache.org/jira/browse/HBASE-24871) | Replication may loss data when refresh recovered replication sources | Major | Replication | @@ -129,6 +135,7 @@ | JIRA | Summary | Priority | Component | |: |: | :--- |: | +| [HBASE-24689](https://issues.apache.org/jira/browse/HBASE-24689) | Generate CHANGES.md and RELEASENOTES.md for 2.2.6 | Major | . | | [HBASE-24881](https://issues.apache.org/jira/browse/HBASE-24881) | Fix flaky TestMasterAbortAndRSGotKilled for branch-2.2 | Major | . | | [HBASE-24870](https://issues.apache.org/jira/browse/HBASE-24870) | Ignore TestAsyncTableRSCrashPublish | Major | . | | [HBASE-24946](https://issues.apache.org/jira/browse/HBASE-24946) | Remove the metrics assert in TestClusterRestartFailover | Major | . | @@ -150,7 +157,6 @@ | [HBASE-24819](https://issues.apache.org/jira/browse/HBASE-24819) | Fix flaky test TestRaceBetweenSCPAndDTP and TestRaceBetweenSCPAndTRSP for branch-2.2 | Major | . | | [HBASE-24756](https://issues.apache.org/jira/browse/HBASE-24756) | Backport HBASE-24336 to branch-2.2 | Minor | metrics | | [HBASE-24690](https://issues.apache.org/jira/browse/HBASE-24690) | Set version to 2.2.6 in branch-2.2 for first RC of 2.2.6 | Major | . | -| [HBASE-24689](https://issues.apache.org/jira/browse/HBASE-24689) | Generate CHANGES.md and RELEASENOTES.md for 2.2.6 | Major | . | | [HBASE-24630](https://issues.apache.org/jira/browse/HBASE-24630) | Purge dev javadoc from client bin tarball | Major | build | | [HBASE-24598](https://issues.apache.org/jira/browse/HBASE-24598) | Port HBASE-24380 (Improve WAL splitting log lines to enable sessionization) to branch-2.2 | Major | logging, Operability, wal | | [HBASE-24347](https://issues.apache.org/jira/browse/HBASE-24347) | Hadoop2 profiles are both active when pre-commit PR builds run | Major | build | diff --git a/RELEASENOTES.md b/RELEASENOTES.md index
[hbase] branch branch-2.2 updated: HBASE-24656 [Flakey Tests] branch-2 TestMasterNoCluster.testStopDuringStart (#1998)
This is an automated email from the ASF dual-hosted git repository.
zghao pushed a commit to branch branch-2.2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.2 by this push:
new 23f295d HBASE-24656 [Flakey Tests] branch-2
TestMasterNoCluster.testStopDuringStart (#1998)
23f295d is described below
commit 23f295d9f271f8d991936b01d41dc97c70fa82eb
Author: Michael Stack
AuthorDate: Mon Jun 29 11:10:14 2020 -0700
HBASE-24656 [Flakey Tests] branch-2 TestMasterNoCluster.testStopDuringStart
(#1998)
---
.../hadoop/hbase/master/TestMasterNoCluster.java | 18 +-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java
index 15b14b4..7322836 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java
@@ -34,6 +34,7 @@ import org.apache.hadoop.hbase.ServerLoad;
import org.apache.hadoop.hbase.ServerMetricsBuilder;
import org.apache.hadoop.hbase.ServerName;
import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.Waiter;
import org.apache.hadoop.hbase.ZooKeeperConnectionException;
import org.apache.hadoop.hbase.client.ClusterConnection;
import org.apache.hadoop.hbase.client.HConnectionTestingUtility;
@@ -117,7 +118,22 @@ public class TestMasterNoCluster {
return false;
}
});
-ZKUtil.deleteNodeRecursively(zkw, zkw.getZNodePaths().baseZNode);
+// Before fails sometimes so retry.
+try {
+ TESTUTIL.waitFor(1, new Waiter.Predicate() {
+@Override public boolean evaluate() throws Exception {
+ try {
+ZKUtil.deleteNodeRecursively(zkw, zkw.getZNodePaths().baseZNode);
+return true;
+ } catch (KeeperException.NotEmptyException e) {
+LOG.info("Failed delete, retrying", e);
+ }
+ return false;
+}
+ });
+} catch (Exception e) {
+ LOG.info("Failed zk clear", e);
+}
zkw.close();
}
[hbase] branch master updated (c2721c8 -> 19d0140)
This is an automated email from the ASF dual-hosted git repository. zghao pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hbase.git. from c2721c8 HBASE-24759 Refuse to update configuration of default group (#2126) add 19d0140 HBASE-24980 Fix dead links in HBase book (#2349) No new revisions were added by this update. Summary of changes: src/main/asciidoc/_chapters/architecture.adoc | 2 +- src/main/asciidoc/_chapters/other_info.adoc | 3 +-- src/site/asciidoc/metrics.adoc| 4 ++-- 3 files changed, 4 insertions(+), 5 deletions(-)
[hbase] branch master updated (19b8a2a -> c2721c8)
This is an automated email from the ASF dual-hosted git repository. zghao pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hbase.git. from 19b8a2a HBASE-19352 Port HADOOP-10379: Protect authentication cookies with the HttpOnly and Secure flags (#2348) add c2721c8 HBASE-24759 Refuse to update configuration of default group (#2126) No new revisions were added by this update. Summary of changes: .../apache/hadoop/hbase/rsgroup/RSGroupInfo.java | 11 +++--- .../hbase/rsgroup/RSGroupInfoManagerImpl.java | 8 - .../hadoop/hbase/rsgroup/TestRSGroupConfig.java| 42 +- 3 files changed, 30 insertions(+), 31 deletions(-)
[hbase] branch branch-2 updated: HBASE-19352 Port HADOOP-10379: Protect authentication cookies with the HttpOnly and Secure flags
This is an automated email from the ASF dual-hosted git repository.
esteban pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new 800a4d9 HBASE-19352 Port HADOOP-10379: Protect authentication cookies
with the HttpOnly and Secure flags
800a4d9 is described below
commit 800a4d9868a0f0eec4ca203e8f0f4d12a54f9d8e
Author: Esteban Gutierrez
AuthorDate: Thu Sep 3 13:20:44 2020 -0500
HBASE-19352 Port HADOOP-10379: Protect authentication cookies with the
HttpOnly and Secure flags
Signed-off-by: Sean Busbey
---
.../org/apache/hadoop/hbase/http/HttpServer.java | 2 +
.../hadoop/hbase/http/TestHttpCookieFlag.java | 191 +
2 files changed, 193 insertions(+)
diff --git
a/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java
b/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java
index 50a6fe5..8a47ca9 100644
--- a/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java
+++ b/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java
@@ -857,6 +857,8 @@ public class HttpServer implements FilterContainer {
fmap.setFilterName(AdminAuthorizedFilter.class.getSimpleName());
webAppContext.getServletHandler().addFilter(filter, fmap);
}
+
webAppContext.getSessionHandler().getSessionCookieConfig().setHttpOnly(true);
+webAppContext.getSessionHandler().getSessionCookieConfig().setSecure(true);
webAppContext.addServlet(holder, pathSpec);
}
diff --git
a/hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestHttpCookieFlag.java
b/hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestHttpCookieFlag.java
new file mode 100644
index 000..d373d60
--- /dev/null
+++
b/hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestHttpCookieFlag.java
@@ -0,0 +1,191 @@
+/**
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License. See accompanying LICENSE file.
+ */
+package org.apache.hadoop.hbase.http;
+
+import java.util.List;
+import java.io.File;
+import java.io.IOException;
+import java.net.HttpURLConnection;
+import java.net.HttpCookie;
+import java.net.URI;
+import java.net.URL;
+import javax.net.ssl.HttpsURLConnection;
+import javax.servlet.Filter;
+import javax.servlet.FilterConfig;
+import javax.servlet.FilterChain;
+import javax.servlet.ServletRequest;
+import javax.servlet.ServletResponse;
+import javax.servlet.ServletException;
+import javax.servlet.http.HttpServletResponse;
+import java.security.GeneralSecurityException;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.testclassification.MiscTests;
+import org.apache.hadoop.hbase.testclassification.SmallTests;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileUtil;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.security.authentication.server.AuthenticationFilter;
+import org.apache.hadoop.security.ssl.KeyStoreTestUtil;
+import org.apache.hadoop.security.ssl.SSLFactory;
+
+import org.junit.Assert;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+@Category({ MiscTests.class, SmallTests.class})
+public class TestHttpCookieFlag {
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+ HBaseClassTestRule.forClass(TestHttpCookieFlag.class);
+
+ private static final String BASEDIR = System.getProperty("test.build.dir",
+ "target/test-dir") + "/" +
+ org.apache.hadoop.hbase.http.TestHttpCookieFlag.class.getSimpleName();
+ private static String keystoresDir;
+ private static String sslConfDir;
+ private static SSLFactory clientSslFactory;
+ private static HttpServer server;
+
+ public static class DummyAuthenticationFilter implements Filter {
+
+@Override
+public void init(FilterConfig filterConfig) throws ServletException {
+}
+
+@Override
+public void doFilter(ServletRequest request, ServletResponse response,
+ FilterChain chain) throws IOException,
+ ServletException {
+ HttpServletResponse resp = (HttpServletResponse) response;
+ boolean isHttps = "https".equals(request.getScheme());
+ AuthenticationFilter.createAuthCookie(resp, "token", null, null, -1,
+
[hbase] branch branch-2.3 updated: HBASE-19352 Port HADOOP-10379: Protect authentication cookies with the HttpOnly and Secure flags
This is an automated email from the ASF dual-hosted git repository.
esteban pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.3 by this push:
new b9fcd1c HBASE-19352 Port HADOOP-10379: Protect authentication cookies
with the HttpOnly and Secure flags
b9fcd1c is described below
commit b9fcd1c4f2b4b1bc959718d631e83c87f4d5deb2
Author: Esteban Gutierrez
AuthorDate: Thu Sep 3 13:20:44 2020 -0500
HBASE-19352 Port HADOOP-10379: Protect authentication cookies with the
HttpOnly and Secure flags
Signed-off-by: Sean Busbey
---
.../org/apache/hadoop/hbase/http/HttpServer.java | 4 +
.../hadoop/hbase/http/TestHttpCookieFlag.java | 191 +
2 files changed, 195 insertions(+)
diff --git
a/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java
b/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java
index d3176ca..f3c4a59 100644
--- a/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java
+++ b/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java
@@ -858,6 +858,10 @@ public class HttpServer implements FilterContainer {
fmap.setFilterName(AdminAuthorizedFilter.class.getSimpleName());
webAppContext.getServletHandler().addFilter(filter, fmap);
}
+webAppContext.getSessionHandler().getSessionManager().
+getSessionCookieConfig().setHttpOnly(true);
+webAppContext.getSessionHandler().getSessionManager().
+getSessionCookieConfig().setSecure(true);
webAppContext.addServlet(holder, pathSpec);
}
diff --git
a/hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestHttpCookieFlag.java
b/hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestHttpCookieFlag.java
new file mode 100644
index 000..d373d60
--- /dev/null
+++
b/hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestHttpCookieFlag.java
@@ -0,0 +1,191 @@
+/**
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License. See accompanying LICENSE file.
+ */
+package org.apache.hadoop.hbase.http;
+
+import java.util.List;
+import java.io.File;
+import java.io.IOException;
+import java.net.HttpURLConnection;
+import java.net.HttpCookie;
+import java.net.URI;
+import java.net.URL;
+import javax.net.ssl.HttpsURLConnection;
+import javax.servlet.Filter;
+import javax.servlet.FilterConfig;
+import javax.servlet.FilterChain;
+import javax.servlet.ServletRequest;
+import javax.servlet.ServletResponse;
+import javax.servlet.ServletException;
+import javax.servlet.http.HttpServletResponse;
+import java.security.GeneralSecurityException;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.testclassification.MiscTests;
+import org.apache.hadoop.hbase.testclassification.SmallTests;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileUtil;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.security.authentication.server.AuthenticationFilter;
+import org.apache.hadoop.security.ssl.KeyStoreTestUtil;
+import org.apache.hadoop.security.ssl.SSLFactory;
+
+import org.junit.Assert;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+@Category({ MiscTests.class, SmallTests.class})
+public class TestHttpCookieFlag {
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+ HBaseClassTestRule.forClass(TestHttpCookieFlag.class);
+
+ private static final String BASEDIR = System.getProperty("test.build.dir",
+ "target/test-dir") + "/" +
+ org.apache.hadoop.hbase.http.TestHttpCookieFlag.class.getSimpleName();
+ private static String keystoresDir;
+ private static String sslConfDir;
+ private static SSLFactory clientSslFactory;
+ private static HttpServer server;
+
+ public static class DummyAuthenticationFilter implements Filter {
+
+@Override
+public void init(FilterConfig filterConfig) throws ServletException {
+}
+
+@Override
+public void doFilter(ServletRequest request, ServletResponse response,
+ FilterChain chain) throws IOException,
+ ServletException {
+ HttpServletResponse resp = (HttpServletResponse) response;
+ boolean isHttps = "https".equals(request.getScheme());
+
[hbase] branch branch-2.2 updated: HBASE-19352 Port HADOOP-10379: Protect authentication cookies with the HttpOnly and Secure flags
This is an automated email from the ASF dual-hosted git repository.
esteban pushed a commit to branch branch-2.2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.2 by this push:
new d234f1f HBASE-19352 Port HADOOP-10379: Protect authentication cookies
with the HttpOnly and Secure flags
d234f1f is described below
commit d234f1fbcc7a34bef57198742d0fc63e5ebf18c8
Author: Esteban Gutierrez
AuthorDate: Thu Sep 3 13:20:44 2020 -0500
HBASE-19352 Port HADOOP-10379: Protect authentication cookies with the
HttpOnly and Secure flags
Signed-off-by: Sean Busbey
---
.../org/apache/hadoop/hbase/http/HttpServer.java | 4 +
.../hadoop/hbase/http/TestHttpCookieFlag.java | 191 +
2 files changed, 195 insertions(+)
diff --git
a/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java
b/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java
index 788fcfb..1e4adee 100644
--- a/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java
+++ b/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java
@@ -823,6 +823,10 @@ public class HttpServer implements FilterContainer {
fmap.setFilterName(AdminAuthorizedFilter.class.getSimpleName());
webAppContext.getServletHandler().addFilter(filter, fmap);
}
+webAppContext.getSessionHandler().getSessionManager().
+getSessionCookieConfig().setHttpOnly(true);
+webAppContext.getSessionHandler().getSessionManager().
+getSessionCookieConfig().setSecure(true);
webAppContext.addServlet(holder, pathSpec);
}
diff --git
a/hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestHttpCookieFlag.java
b/hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestHttpCookieFlag.java
new file mode 100644
index 000..d373d60
--- /dev/null
+++
b/hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestHttpCookieFlag.java
@@ -0,0 +1,191 @@
+/**
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License. See accompanying LICENSE file.
+ */
+package org.apache.hadoop.hbase.http;
+
+import java.util.List;
+import java.io.File;
+import java.io.IOException;
+import java.net.HttpURLConnection;
+import java.net.HttpCookie;
+import java.net.URI;
+import java.net.URL;
+import javax.net.ssl.HttpsURLConnection;
+import javax.servlet.Filter;
+import javax.servlet.FilterConfig;
+import javax.servlet.FilterChain;
+import javax.servlet.ServletRequest;
+import javax.servlet.ServletResponse;
+import javax.servlet.ServletException;
+import javax.servlet.http.HttpServletResponse;
+import java.security.GeneralSecurityException;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.testclassification.MiscTests;
+import org.apache.hadoop.hbase.testclassification.SmallTests;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileUtil;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.security.authentication.server.AuthenticationFilter;
+import org.apache.hadoop.security.ssl.KeyStoreTestUtil;
+import org.apache.hadoop.security.ssl.SSLFactory;
+
+import org.junit.Assert;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+@Category({ MiscTests.class, SmallTests.class})
+public class TestHttpCookieFlag {
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+ HBaseClassTestRule.forClass(TestHttpCookieFlag.class);
+
+ private static final String BASEDIR = System.getProperty("test.build.dir",
+ "target/test-dir") + "/" +
+ org.apache.hadoop.hbase.http.TestHttpCookieFlag.class.getSimpleName();
+ private static String keystoresDir;
+ private static String sslConfDir;
+ private static SSLFactory clientSslFactory;
+ private static HttpServer server;
+
+ public static class DummyAuthenticationFilter implements Filter {
+
+@Override
+public void init(FilterConfig filterConfig) throws ServletException {
+}
+
+@Override
+public void doFilter(ServletRequest request, ServletResponse response,
+ FilterChain chain) throws IOException,
+ ServletException {
+ HttpServletResponse resp = (HttpServletResponse) response;
+ boolean isHttps = "https".equals(request.getScheme());
+
[hbase-site] branch asf-site updated: INFRA-10751 Empty commit
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hbase-site.git The following commit(s) were added to refs/heads/asf-site by this push: new 64fc90b INFRA-10751 Empty commit 64fc90b is described below commit 64fc90b9873707c29eef1f7ff07abf82996f913d Author: jenkins AuthorDate: Thu Sep 3 20:21:36 2020 + INFRA-10751 Empty commit
[hbase] branch revert-2348-HBASE-19352 created (now 1c1a7be)
This is an automated email from the ASF dual-hosted git repository. esteban pushed a change to branch revert-2348-HBASE-19352 in repository https://gitbox.apache.org/repos/asf/hbase.git. at 1c1a7be Revert "HBASE-19352 Port HADOOP-10379: Protect authentication cookies with the HttpOnly and Secure flags (#2348)" This branch includes the following new commits: new 1c1a7be Revert "HBASE-19352 Port HADOOP-10379: Protect authentication cookies with the HttpOnly and Secure flags (#2348)" The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[hbase] 01/01: Revert " HBASE-19352 Port HADOOP-10379: Protect authentication cookies with the HttpOnly and Secure flags (#2348)"
This is an automated email from the ASF dual-hosted git repository.
esteban pushed a commit to branch revert-2348-HBASE-19352
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 1c1a7be1a63525468a20180a0bb554abb0c8be3f
Author: Esteban Gutierrez
AuthorDate: Thu Sep 3 13:38:45 2020 -0500
Revert "HBASE-19352 Port HADOOP-10379: Protect authentication cookies
with the HttpOnly and Secure flags (#2348)"
This reverts commit 19b8a2a64a63e9e546af3497871b5346ea5b6b5b.
---
.../org/apache/hadoop/hbase/http/HttpServer.java | 2 -
.../hadoop/hbase/http/TestHttpCookieFlag.java | 191 -
2 files changed, 193 deletions(-)
diff --git
a/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java
b/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java
index 8a47ca9..50a6fe5 100644
--- a/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java
+++ b/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java
@@ -857,8 +857,6 @@ public class HttpServer implements FilterContainer {
fmap.setFilterName(AdminAuthorizedFilter.class.getSimpleName());
webAppContext.getServletHandler().addFilter(filter, fmap);
}
-
webAppContext.getSessionHandler().getSessionCookieConfig().setHttpOnly(true);
-webAppContext.getSessionHandler().getSessionCookieConfig().setSecure(true);
webAppContext.addServlet(holder, pathSpec);
}
diff --git
a/hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestHttpCookieFlag.java
b/hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestHttpCookieFlag.java
deleted file mode 100644
index d373d60..000
---
a/hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestHttpCookieFlag.java
+++ /dev/null
@@ -1,191 +0,0 @@
-/**
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License. See accompanying LICENSE file.
- */
-package org.apache.hadoop.hbase.http;
-
-import java.util.List;
-import java.io.File;
-import java.io.IOException;
-import java.net.HttpURLConnection;
-import java.net.HttpCookie;
-import java.net.URI;
-import java.net.URL;
-import javax.net.ssl.HttpsURLConnection;
-import javax.servlet.Filter;
-import javax.servlet.FilterConfig;
-import javax.servlet.FilterChain;
-import javax.servlet.ServletRequest;
-import javax.servlet.ServletResponse;
-import javax.servlet.ServletException;
-import javax.servlet.http.HttpServletResponse;
-import java.security.GeneralSecurityException;
-import org.apache.hadoop.hbase.HBaseClassTestRule;
-import org.apache.hadoop.hbase.testclassification.MiscTests;
-import org.apache.hadoop.hbase.testclassification.SmallTests;
-import org.apache.hadoop.conf.Configuration;
-import org.apache.hadoop.fs.FileUtil;
-import org.apache.hadoop.net.NetUtils;
-import org.apache.hadoop.security.authentication.server.AuthenticationFilter;
-import org.apache.hadoop.security.ssl.KeyStoreTestUtil;
-import org.apache.hadoop.security.ssl.SSLFactory;
-
-import org.junit.Assert;
-import org.junit.AfterClass;
-import org.junit.BeforeClass;
-import org.junit.ClassRule;
-import org.junit.Test;
-import org.junit.experimental.categories.Category;
-
-@Category({ MiscTests.class, SmallTests.class})
-public class TestHttpCookieFlag {
- @ClassRule
- public static final HBaseClassTestRule CLASS_RULE =
- HBaseClassTestRule.forClass(TestHttpCookieFlag.class);
-
- private static final String BASEDIR = System.getProperty("test.build.dir",
- "target/test-dir") + "/" +
- org.apache.hadoop.hbase.http.TestHttpCookieFlag.class.getSimpleName();
- private static String keystoresDir;
- private static String sslConfDir;
- private static SSLFactory clientSslFactory;
- private static HttpServer server;
-
- public static class DummyAuthenticationFilter implements Filter {
-
-@Override
-public void init(FilterConfig filterConfig) throws ServletException {
-}
-
-@Override
-public void doFilter(ServletRequest request, ServletResponse response,
- FilterChain chain) throws IOException,
- ServletException {
- HttpServletResponse resp = (HttpServletResponse) response;
- boolean isHttps = "https".equals(request.getScheme());
- AuthenticationFilter.createAuthCookie(resp, "token", null, null, -1,
- true, isHttps);
- chain.doFilter(request, resp);
-}
-
-@Override
-public void destroy() {
-}
- }
- public static class
[hbase] branch master updated (a352706 -> 19b8a2a)
This is an automated email from the ASF dual-hosted git repository. esteban pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hbase.git. from a352706 HBASE-24940: runCatalogJanitor() API should return -1 to indicate already running status add 19b8a2a HBASE-19352 Port HADOOP-10379: Protect authentication cookies with the HttpOnly and Secure flags (#2348) No new revisions were added by this update. Summary of changes: .../org/apache/hadoop/hbase/http/HttpServer.java | 2 + .../hadoop/hbase/http/TestHttpCookieFlag.java | 191 + 2 files changed, 193 insertions(+) create mode 100644 hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestHttpCookieFlag.java
[hbase] branch branch-1 updated: HBASE-24940: runCatalogJanitor() API should return -1 to indicate already running status
This is an automated email from the ASF dual-hosted git repository.
vjasani pushed a commit to branch branch-1
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-1 by this push:
new 76db1e1 HBASE-24940: runCatalogJanitor() API should return -1 to
indicate already running status
76db1e1 is described below
commit 76db1e1791cfa2a6685580af254a8180363a8dba
Author: Mohammad Arshad
AuthorDate: Thu Sep 3 15:46:10 2020 +0530
HBASE-24940: runCatalogJanitor() API should return -1 to indicate already
running status
Closes #2331
Co-authored-by: Viraj Jasani
Signed-off-by: Viraj Jasani
---
.../java/org/apache/hadoop/hbase/client/Admin.java | 2 +-
.../apache/hadoop/hbase/master/CatalogJanitor.java | 4 ++-
.../hadoop/hbase/master/TestCatalogJanitor.java| 32 ++
3 files changed, 36 insertions(+), 2 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
index 4896df9..c32b391 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
@@ -779,7 +779,7 @@ public interface Admin extends Abortable, Closeable {
/**
* Ask for a scan of the catalog table
*
- * @return the number of entries cleaned
+ * @return the number of entries cleaned. Returns -1 if previous run is in
progress.
* @throws IOException if a remote or network exception occurs
*/
int runCatalogScan() throws IOException;
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
index 549c38f..911248c 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
@@ -238,7 +238,9 @@ public class CatalogJanitor extends ScheduledChore {
int scan() throws IOException {
try {
if (!alreadyRunning.compareAndSet(false, true)) {
-return 0;
+LOG.debug("CatalogJanitor already running");
+// -1 indicates previous scan is in progress
+return -1;
}
Triple, Map>
scanTriple =
getMergedRegionsAndSplitParents();
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
index 9416620..397d5a8 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
@@ -26,6 +26,7 @@ import static org.mockito.Mockito.doReturn;
import static org.mockito.Mockito.spy;
import java.io.IOException;
+import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.SortedMap;
@@ -1068,6 +1069,37 @@ public class TestCatalogJanitor {
janitor.cancel(true);
}
+ @Test
+ public void testAlreadyRunningStatus() throws Exception {
+int numberOfThreads = 2;
+final List gcValues = new ArrayList<>();
+Thread[] threads = new Thread[numberOfThreads];
+HBaseTestingUtility hBaseTestingUtility = new HBaseTestingUtility();
+
hBaseTestingUtility.getConfiguration().setInt("hbase.client.retries.number", 5);
+Server server = new MockServer(hBaseTestingUtility);
+MasterServices services = new MockMasterServices(server);
+final CatalogJanitor catalogJanitor = new CatalogJanitor(server, services);
+for (int i = 0; i < numberOfThreads; i++) {
+ threads[i] = new Thread(new Runnable() {
+@Override
+public void run() {
+ try {
+gcValues.add(catalogJanitor.scan());
+ } catch (IOException e) {
+throw new RuntimeException(e);
+ }
+}
+ });
+}
+for (int i = 0; i < numberOfThreads; i++) {
+ threads[i].start();
+}
+for (int i = 0; i < numberOfThreads; i++) {
+ threads[i].join();
+}
+assertTrue("One janitor.scan() call should have returned -1",
gcValues.contains(-1));
+ }
+
private FileStatus[] addMockStoreFiles(int count, MasterServices services,
Path storedir)
throws IOException {
// get the existing store files
[hbase] branch branch-2.3 updated: HBASE-24940: runCatalogJanitor() API should return -1 to indicate already running status
This is an automated email from the ASF dual-hosted git repository.
vjasani pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.3 by this push:
new dc79b26 HBASE-24940: runCatalogJanitor() API should return -1 to
indicate already running status
dc79b26 is described below
commit dc79b267618c3c05ae0cb26c6be925abb415bc5e
Author: Mohammad Arshad
AuthorDate: Thu Sep 3 15:46:10 2020 +0530
HBASE-24940: runCatalogJanitor() API should return -1 to indicate already
running status
Closes #2331
Signed-off-by: Viraj Jasani
---
.../java/org/apache/hadoop/hbase/client/Admin.java | 2 +-
.../apache/hadoop/hbase/master/CatalogJanitor.java | 4 +++-
.../hadoop/hbase/master/TestCatalogJanitor.java| 25 ++
3 files changed, 29 insertions(+), 2 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
index d34f56f..75c71a3 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
@@ -1333,7 +1333,7 @@ public interface Admin extends Abortable, Closeable {
/**
* Ask for a scan of the catalog table.
*
- * @return the number of entries cleaned
+ * @return the number of entries cleaned. Returns -1 if previous run is in
progress.
* @throws IOException if a remote or network exception occurs
* @deprecated Since 2.0.0. Will be removed in 3.0.0. Use {@link
#runCatalogJanitor()}}
* instead.
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
index af413f3..33ba382 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
@@ -160,6 +160,7 @@ public class CatalogJanitor extends ScheduledChore {
* Run janitorial scan of catalog hbase:meta table looking for
* garbage to collect.
* @return How many items gc'd whether for merge or split.
+ * Returns -1 if previous scan is in progress.
*/
@VisibleForTesting
public int scan() throws IOException {
@@ -167,7 +168,8 @@ public class CatalogJanitor extends ScheduledChore {
try {
if (!alreadyRunning.compareAndSet(false, true)) {
LOG.debug("CatalogJanitor already running");
-return gcs;
+// -1 indicates previous scan is in progress
+return -1;
}
this.lastReport = scanForReport();
if (!this.lastReport.isEmpty()) {
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
index 97a0f6f..5486817 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
@@ -25,6 +25,8 @@ import static org.mockito.Mockito.doReturn;
import static org.mockito.Mockito.spy;
import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
import java.util.Map;
import java.util.NavigableMap;
import java.util.Objects;
@@ -555,6 +557,29 @@ public class TestCatalogJanitor {
assertArchiveEqualToOriginal(storeFiles, archivedStoreFiles, fs, true);
}
+ @Test
+ public void testAlreadyRunningStatus() throws Exception {
+int numberOfThreads = 2;
+List gcValues = new ArrayList<>();
+Thread[] threads = new Thread[numberOfThreads];
+for (int i = 0; i < numberOfThreads; i++) {
+ threads[i] = new Thread(() -> {
+try {
+ gcValues.add(janitor.scan());
+} catch (IOException e) {
+ throw new RuntimeException(e);
+}
+ });
+}
+for (int i = 0; i < numberOfThreads; i++) {
+ threads[i].start();
+}
+for (int i = 0; i < numberOfThreads; i++) {
+ threads[i].join();
+}
+assertTrue("One janitor.scan() call should have returned -1",
gcValues.contains(-1));
+ }
+
private FileStatus[] addMockStoreFiles(int count, MasterServices services,
Path storedir)
throws IOException {
// get the existing store files
[hbase] branch branch-2 updated: HBASE-24940: runCatalogJanitor() API should return -1 to indicate already running status
This is an automated email from the ASF dual-hosted git repository.
vjasani pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new 17eeaef HBASE-24940: runCatalogJanitor() API should return -1 to
indicate already running status
17eeaef is described below
commit 17eeaef6d96f3d70fba26f37216b75efc35b987e
Author: Mohammad Arshad
AuthorDate: Thu Sep 3 15:46:10 2020 +0530
HBASE-24940: runCatalogJanitor() API should return -1 to indicate already
running status
Closes #2331
Signed-off-by: Viraj Jasani
---
.../java/org/apache/hadoop/hbase/client/Admin.java | 2 +-
.../apache/hadoop/hbase/master/CatalogJanitor.java | 4 +++-
.../hadoop/hbase/master/TestCatalogJanitor.java| 25 ++
3 files changed, 29 insertions(+), 2 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
index 20b1fc0..d76ca51 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
@@ -1366,7 +1366,7 @@ public interface Admin extends Abortable, Closeable {
/**
* Ask for a scan of the catalog table.
*
- * @return the number of entries cleaned
+ * @return the number of entries cleaned. Returns -1 if previous run is in
progress.
* @throws IOException if a remote or network exception occurs
* @deprecated Since 2.0.0. Will be removed in 3.0.0. Use {@link
#runCatalogJanitor()}}
* instead.
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
index af413f3..33ba382 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
@@ -160,6 +160,7 @@ public class CatalogJanitor extends ScheduledChore {
* Run janitorial scan of catalog hbase:meta table looking for
* garbage to collect.
* @return How many items gc'd whether for merge or split.
+ * Returns -1 if previous scan is in progress.
*/
@VisibleForTesting
public int scan() throws IOException {
@@ -167,7 +168,8 @@ public class CatalogJanitor extends ScheduledChore {
try {
if (!alreadyRunning.compareAndSet(false, true)) {
LOG.debug("CatalogJanitor already running");
-return gcs;
+// -1 indicates previous scan is in progress
+return -1;
}
this.lastReport = scanForReport();
if (!this.lastReport.isEmpty()) {
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
index 97a0f6f..5486817 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
@@ -25,6 +25,8 @@ import static org.mockito.Mockito.doReturn;
import static org.mockito.Mockito.spy;
import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
import java.util.Map;
import java.util.NavigableMap;
import java.util.Objects;
@@ -555,6 +557,29 @@ public class TestCatalogJanitor {
assertArchiveEqualToOriginal(storeFiles, archivedStoreFiles, fs, true);
}
+ @Test
+ public void testAlreadyRunningStatus() throws Exception {
+int numberOfThreads = 2;
+List gcValues = new ArrayList<>();
+Thread[] threads = new Thread[numberOfThreads];
+for (int i = 0; i < numberOfThreads; i++) {
+ threads[i] = new Thread(() -> {
+try {
+ gcValues.add(janitor.scan());
+} catch (IOException e) {
+ throw new RuntimeException(e);
+}
+ });
+}
+for (int i = 0; i < numberOfThreads; i++) {
+ threads[i].start();
+}
+for (int i = 0; i < numberOfThreads; i++) {
+ threads[i].join();
+}
+assertTrue("One janitor.scan() call should have returned -1",
gcValues.contains(-1));
+ }
+
private FileStatus[] addMockStoreFiles(int count, MasterServices services,
Path storedir)
throws IOException {
// get the existing store files
[hbase] branch master updated: HBASE-24940: runCatalogJanitor() API should return -1 to indicate already running status
This is an automated email from the ASF dual-hosted git repository.
vjasani pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new a352706 HBASE-24940: runCatalogJanitor() API should return -1 to
indicate already running status
a352706 is described below
commit a3527067004e7496dd892f37a2ebcd5b76a71e26
Author: Mohammad Arshad
AuthorDate: Thu Sep 3 15:46:10 2020 +0530
HBASE-24940: runCatalogJanitor() API should return -1 to indicate already
running status
Closes #2331
Signed-off-by: Viraj Jasani
---
.../java/org/apache/hadoop/hbase/client/Admin.java | 2 +-
.../apache/hadoop/hbase/master/CatalogJanitor.java | 4 +++-
.../hadoop/hbase/master/TestCatalogJanitor.java| 25 ++
3 files changed, 29 insertions(+), 2 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
index 40db1c1..f4fd14a 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
@@ -903,7 +903,7 @@ public interface Admin extends Abortable, Closeable {
/**
* Ask for a scan of the catalog table.
*
- * @return the number of entries cleaned
+ * @return the number of entries cleaned. Returns -1 if previous run is in
progress.
* @throws IOException if a remote or network exception occurs
*/
int runCatalogJanitor() throws IOException;
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
index 9e9a00f..b860d64 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
@@ -162,6 +162,7 @@ public class CatalogJanitor extends ScheduledChore {
* Run janitorial scan of catalog hbase:meta table looking for
* garbage to collect.
* @return How many items gc'd whether for merge or split.
+ * Returns -1 if previous scan is in progress.
*/
@VisibleForTesting
public int scan() throws IOException {
@@ -169,7 +170,8 @@ public class CatalogJanitor extends ScheduledChore {
try {
if (!alreadyRunning.compareAndSet(false, true)) {
LOG.debug("CatalogJanitor already running");
-return gcs;
+// -1 indicates previous scan is in progress
+return -1;
}
this.lastReport = scanForReport();
if (!this.lastReport.isEmpty()) {
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
index 2e668f3..695b8b9 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
@@ -25,6 +25,8 @@ import static org.mockito.Mockito.doReturn;
import static org.mockito.Mockito.spy;
import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
import java.util.Map;
import java.util.NavigableMap;
import java.util.Objects;
@@ -551,6 +553,29 @@ public class TestCatalogJanitor {
assertArchiveEqualToOriginal(storeFiles, archivedStoreFiles, fs, true);
}
+ @Test
+ public void testAlreadyRunningStatus() throws Exception {
+int numberOfThreads = 2;
+List gcValues = new ArrayList<>();
+Thread[] threads = new Thread[numberOfThreads];
+for (int i = 0; i < numberOfThreads; i++) {
+ threads[i] = new Thread(() -> {
+try {
+ gcValues.add(janitor.scan());
+} catch (IOException e) {
+ throw new RuntimeException(e);
+}
+ });
+}
+for (int i = 0; i < numberOfThreads; i++) {
+ threads[i].start();
+}
+for (int i = 0; i < numberOfThreads; i++) {
+ threads[i].join();
+}
+assertTrue("One janitor.scan() call should have returned -1",
gcValues.contains(-1));
+ }
+
private FileStatus[] addMockStoreFiles(int count, MasterServices services,
Path storedir)
throws IOException {
// get the existing store files
[hbase] branch branch-1 updated (1dc75e7 -> a48515a)
This is an automated email from the ASF dual-hosted git repository. vjasani pushed a change to branch branch-1 in repository https://gitbox.apache.org/repos/asf/hbase.git. from 1dc75e7 [HBASE-24893] Fix failing TestLogLevel Unit Test (#2338) add a48515a HBASE-24970 Backport HBASE-20985 to branch-1 No new revisions were added by this update. Summary of changes: .../org/apache/hadoop/hbase/HTableDescriptor.java | 36 ++ .../master/normalizer/SimpleRegionNormalizer.java | 32 - .../normalizer/TestSimpleRegionNormalizer.java | 131 ++--- hbase-shell/src/main/ruby/hbase/admin.rb | 12 +- hbase-shell/src/main/ruby/shell/commands/alter.rb | 3 +- hbase-shell/src/test/ruby/hbase/admin_test.rb | 7 ++ 6 files changed, 200 insertions(+), 21 deletions(-)
