Request for assistance with September branch-1.3 and branch-1.4 release candidates
I have been pretty busy with work and life so haven't had the bandwidth to make the next set of 1.3 and 1.4 releases, nor another 1.5.0 candidate either. I have started a few times. I will get as far as observing some issues still in progress targeting the respective release(s) on JIRA, or finding that looped unit suite execution reveals a flaky unit test or two deserving of a bit of investigation. Next week I am heading out for vacation for two weeks. Therefore It is unlikely I will have any time in the entire month of September for branch-1.3, branch-1.4, or branch-1 release candidates. If someone else would like to contribute time and attention to release management, especially for branch-1.3 (1.3.6) and branch-1.4 (1.4.11), that would be great. Both of these branches need releasing to get the bug fix out for those code lines where recovered.edits files may be misplaced when root FS != wal FS. If not I'll spin a set when I return, end of September maybe, or early October. -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
[jira] [Resolved] (HBASE-21447) HBCK2 tool have questions on holes when HBCK2 checks region chain
[
https://issues.apache.org/jira/browse/HBASE-21447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-21447.
---
Resolution: Cannot Reproduce
[~nicholasjiang] I'm resolving as 'cannot reproduce' mostly because a bunch of
fixes have gone into hbase and hbck2 including a 'fix holes'. I afraid it means
upgrade to later versions of hbase so new methods on hbck service are
available. Thanks for reporting.
> HBCK2 tool have questions on holes when HBCK2 checks region chain
> ---
>
> Key: HBASE-21447
> URL: https://issues.apache.org/jira/browse/HBASE-21447
> Project: HBase
> Issue Type: Improvement
> Components: hbck2
>Affects Versions: 2.0.2
>Reporter: Nicholas Jiang
>Priority: Major
> Attachments: Hole.png
>
>
> [hbck2]https://github.com/apache/hbase-operator-tools/tree/master/hbase-hbck2
> This HBCK2 tool have some questions on holes when HBCK2 checks region chain
> as follows.
> {code:java}
> ERROR: There is a hole in the region chain between \x01F\x00\x00 and
> \x02\x8C\x00\x00. You need to create a new .regioninfo and region dir in hdfs
> to plug the hole.
> ERROR: There is a hole in the region chain between \x05\x18\x00\x00 and
> \x06^\x00\x00. You need to create a new .regioninfo and region dir in hdfs to
> plug the hole.
> ERROR: There is a hole in the region chain between \x07\x01\x00\x00 and
> \x07\xA4\x00\x00. You need to create a new .regioninfo and region dir in hdfs
> to plug the hole.
> ERROR: There is a hole in the region chain between \x08G\x00\x00 and
> \x09\x8D\x00\x00. You need to create a new .regioninfo and region dir in hdfs
> to plug the hole.
> ERROR: There is a hole in the region chain between \x0A0\x00\x00 and
> \x0Bv\x00\x00. You need to create a new .regioninfo and region dir in hdfs to
> plug the hole.
> ERROR: There is a hole in the region chain between \x0C\x19\x00\x00 and
> \x0C\xBC\x00\x00. You need to create a new .regioninfo and region dir in hdfs
> to plug the hole.
> ERROR: There is a hole in the region chain between \x0D_\x00\x00 and
> \x0E\xA5\x00\x00. You need to create a new .regioninfo and region dir in hdfs
> to plug the hole.
> ERROR: There is a hole in the region chain between \x0F\xEB\x00\x00 and
> \x111\x00\x00. You need to create a new .regioninfo and region dir in hdfs to
> plug the hole.
> ERROR: There is a hole in the region chain between \x16I\x00\x00 and
> \x16\xEC\x00\x00. You need to create a new .regioninfo and region dir in hdfs
> to plug the hole.
> ERROR: There is a hole in the region chain between (\xC0\x00\x00 and
> *\x06\x00\x00. You need to create a new .regioninfo and region dir in hdfs to
> plug the hole.
> {code}
> !Hole.png!
> This hole problem can't be solved by HBCK2 tool.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (HBASE-22985) Gracefully handle invalid ServiceLoader entries
Josh Elser created HBASE-22985:
--
Summary: Gracefully handle invalid ServiceLoader entries
Key: HBASE-22985
URL: https://issues.apache.org/jira/browse/HBASE-22985
Project: HBase
Issue Type: Bug
Components: metrics
Reporter: Josh Elser
Assignee: Josh Elser
Just saw this happen: A RegionServer failed to start because, on the classpath,
there was a {{META-INF/services}} entry in a JAR on the classpath that was
advertising an implementation of
{{org.apache.hadoop.hbase.metrics.MetricsRegistries}} but was an implementation
of a completely different class:
{noformat}
Caused by: java.util.ServiceConfigurationError:
org.apache.hadoop.hbase.metrics.MetricRegistries: Provider
org.apache.ratis.metrics.impl.MetricRegistriesImpl not a subtype
at java.util.ServiceLoader.fail(ServiceLoader.java:239)
at java.util.ServiceLoader.access$300(ServiceLoader.java:185)
at
java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:376)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at
org.apache.hadoop.hbase.metrics.MetricRegistriesLoader.getDefinedImplemantations(MetricRegistriesLoader.java:92)
at
org.apache.hadoop.hbase.metrics.MetricRegistriesLoader.load(MetricRegistriesLoader.java:50)
at
org.apache.hadoop.hbase.metrics.MetricRegistries$LazyHolder.(MetricRegistries.java:39)
at
org.apache.hadoop.hbase.metrics.MetricRegistries.global(MetricRegistries.java:47)
at
org.apache.hadoop.hbase.metrics.BaseSourceImpl.(BaseSourceImpl.java:122)
at
org.apache.hadoop.hbase.io.MetricsIOSourceImpl.(MetricsIOSourceImpl.java:46)
at
org.apache.hadoop.hbase.io.MetricsIOSourceImpl.(MetricsIOSourceImpl.java:38)
at
org.apache.hadoop.hbase.regionserver.MetricsRegionServerSourceFactoryImpl.createIO(MetricsRegionServerSourceFactoryImpl.java:84)
at org.apache.hadoop.hbase.io.MetricsIO.(MetricsIO.java:35)
at org.apache.hadoop.hbase.io.hfile.HFile.(HFile.java:195)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:570)
... 10 more{noformat}
Now, we could catch this and gracefully ignore it; however, this would mean
that we're catching an Error which is typically considered a smell.
It's a pretty straightforward change, so I'm apt to think that it's OK. What do
other folks think?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (HBASE-22567) [HBCK2] Add new methods for dealing with missing regions in META while Master is online
[
https://issues.apache.org/jira/browse/HBASE-22567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wellington Chevreuil resolved HBASE-22567.
--
Resolution: Fixed
Committed last PR version.
> [HBCK2] Add new methods for dealing with missing regions in META while Master
> is online
> ---
>
> Key: HBASE-22567
> URL: https://issues.apache.org/jira/browse/HBASE-22567
> Project: HBase
> Issue Type: New Feature
> Components: hbck2
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
>
> Following latest discussion on HBASE-21745, this proposes an hbck2 command
> that allows for inserting back regions missing in META that still have
> *regioninfo* available in HDFS. Although this is still an interactive and
> simpler version than the old _OfflineMetaRepair_, it still relies on hdfs
> state as the source of truth, and performs META updates mostly independently
> from Master (apart from requiring Meta table been online).
> For a more detailed explanation on this command behaviour, pasting _command
> usage_ text:
> {noformat}
> To be used for scenarios where some regions may be missing in META,
> but there's still a valid 'regioninfo' metadata file on HDFS.
> This is a lighter version of 'OfflineMetaRepair' tool commonly used for
> similar issues on 1.x release line.
> This command needs META to be online. For each table name passed as
> parameter, it performs a diff between regions available in META,
> against existing regions dirs on HDFS. Then, for region dirs with
> no matches in META, it reads regioninfo metadata file and
> re-creates given region in META. Regions are re-created in 'CLOSED'
> state at META table only, but not in Masters' cache, and are not
> assigned either. A rolling Masters restart, followed by a
> hbck2 'assigns' command with all re-inserted regions is required.
> This hbck2 'assigns' command is printed for user convenience.
> WARNING: To avoid potential region overlapping problems due to ongoing
> splits, this command disables given tables while re-inserting regions.
> An example adding missing regions for tables 'table_1' and 'table_2':
> $ HBCK2 addMissingRegionsInMeta table_1 table_2
> Returns hbck2 'assigns' command with all re-inserted regions.{noformat}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (HBASE-22984) [HBCK2] HBCKMetaTableAccessor.deleteFromMetaTable throwing java.lang.UnsupportedOperationException at runtime
[
https://issues.apache.org/jira/browse/HBASE-22984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wellington Chevreuil resolved HBASE-22984.
--
Resolution: Fixed
Thanks for quickly reviewing this PR [~busbey]!
> [HBCK2] HBCKMetaTableAccessor.deleteFromMetaTable throwing
> java.lang.UnsupportedOperationException at runtime
> -
>
> Key: HBASE-22984
> URL: https://issues.apache.org/jira/browse/HBASE-22984
> Project: HBase
> Issue Type: Bug
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
>
> Came around this while converting some UTs from HBASE-22567 to use
> *HBCKMetaTableAccessor.* When calling
> *HBCKMetaTableAccessor.deleteRegionInfo,* I'm getting error as shown below,
> because list returned by _Arrays.asList_ apparently does not implement
> _delete_ method:
> {noformat}
> java.lang.UnsupportedOperationException
> at java.util.AbstractList.remove(AbstractList.java:161)
> at org.apache.hadoop.hbase.client.HTable.delete(HTable.java:522)
> at
> org.apache.hbase.HBCKMetaTableAccessor.deleteFromMetaTable(HBCKMetaTableAccessor.java:136)
> at
> org.apache.hbase.HBCKMetaTableAccessor.deleteRegionInfo(HBCKMetaTableAccessor.java:89)
> at
> org.apache.hbase.TestHBCK2.testReportMissingRegionsInMetaSpecificTblAndNsTblAlsoMissing(TestHBCK2.java:225)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (HBASE-22984) [HBCK2] HBCKMetaTableAccessor.deleteFromMetaTable throwing java.lang.UnsupportedOperationException at runtime
Wellington Chevreuil created HBASE-22984:
Summary: [HBCK2] HBCKMetaTableAccessor.deleteFromMetaTable
throwing java.lang.UnsupportedOperationException at runtime
Key: HBASE-22984
URL: https://issues.apache.org/jira/browse/HBASE-22984
Project: HBase
Issue Type: Bug
Reporter: Wellington Chevreuil
Assignee: Wellington Chevreuil
Came around this while converting some UTs from HBASE-22567 to use
*HBCKMetaTableAccessor.* When calling *HBCKMetaTableAccessor.deleteRegionInfo,*
I'm getting error as shown below, because list returned by _Arrays.asList_
apparently does not implement _delete_ method:
{noformat}
java.lang.UnsupportedOperationException
at java.util.AbstractList.remove(AbstractList.java:161)
at org.apache.hadoop.hbase.client.HTable.delete(HTable.java:522)
at
org.apache.hbase.HBCKMetaTableAccessor.deleteFromMetaTable(HBCKMetaTableAccessor.java:136)
at
org.apache.hbase.HBCKMetaTableAccessor.deleteRegionInfo(HBCKMetaTableAccessor.java:89)
at
org.apache.hbase.TestHBCK2.testReportMissingRegionsInMetaSpecificTblAndNsTblAlsoMissing(TestHBCK2.java:225)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (HBASE-22983) [HBCK2] Record executed command and output to log file
Peter Somogyi created HBASE-22983: - Summary: [HBCK2] Record executed command and output to log file Key: HBASE-22983 URL: https://issues.apache.org/jira/browse/HBASE-22983 Project: HBase Issue Type: Sub-task Components: hbase-operator-tools Reporter: Peter Somogyi HBCK2 operations by default are logged to the console only. For troubleshooting and tracking it is helpful to write the logs to a file with the executed command arguments. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (HBASE-22981) Remove unused flags for Yetus
[
https://issues.apache.org/jira/browse/HBASE-22981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi resolved HBASE-22981.
---
Fix Version/s: 2.2.2
2.1.7
1.4.11
1.3.6
2.3.0
1.5.0
3.0.0
Resolution: Fixed
Pushed to all branches. Thanks [~busbey] for the review!
> Remove unused flags for Yetus
> -
>
> Key: HBASE-22981
> URL: https://issues.apache.org/jira/browse/HBASE-22981
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 1.3.6, 1.4.11, 2.1.7
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 2.3.0, 1.3.6, 1.4.11, 2.1.7, 2.2.2
>
>
> Nightly jobs are failing with yetus error.
> {noformat}
> [2019-09-05T23:08:16.473Z] | -1 | yetus | 0m 6s | Unprocessed
> flag(s): --jenkins
> [2019-09-05T23:08:16.473Z] | | ||
> --build-url-patchdir
> {noformat}
> Starting from Yetus 0.11.0 `--ignore-unknown-options=true` flag needs to be
> passed to yetus, otherwise unrecognized flags cause build failures.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (HBASE-22982) Send SIGSTOP to hang or SIGCONT to resume rs and add graceful rolling restart
Szabolcs Bukros created HBASE-22982: --- Summary: Send SIGSTOP to hang or SIGCONT to resume rs and add graceful rolling restart Key: HBASE-22982 URL: https://issues.apache.org/jira/browse/HBASE-22982 Project: HBase Issue Type: Sub-task Components: integration tests Affects Versions: 3.0.0 Reporter: Szabolcs Bukros * Add a Chaos Monkey action that uses SIGSTOP and SIGCONT to hang and resume a ratio of region servers. * Add a Chaos Monkey action to simulate a rolling restart including graceful_stop like functionality that unloads the regions from the server before a restart and then places it under load again afterwards. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (HBASE-22981) Ignore unprocessed flags on nightlies
Peter Somogyi created HBASE-22981:
-
Summary: Ignore unprocessed flags on nightlies
Key: HBASE-22981
URL: https://issues.apache.org/jira/browse/HBASE-22981
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 1.3.6, 1.4.11, 2.1.7
Reporter: Peter Somogyi
Assignee: Peter Somogyi
Nightly jobs are failing with yetus error.
{noformat}
[2019-09-05T23:08:16.473Z] | -1 | yetus | 0m 6s | Unprocessed
flag(s): --jenkins
[2019-09-05T23:08:16.473Z] | | ||
--build-url-patchdir
{noformat}
Starting from Yetus 0.11.0 `--ignore-unknown-options=true` flag needs to be
passed to yetus, otherwise unrecognized flags cause build failures.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (HBASE-22980) HRegionPartioner getPartition() method incorrectly partitions the regions of the table.
Shardul Singh created HBASE-22980:
-
Summary: HRegionPartioner getPartition() method incorrectly
partitions the regions of the table.
Key: HBASE-22980
URL: https://issues.apache.org/jira/browse/HBASE-22980
Project: HBase
Issue Type: Bug
Environment: Instead of
{code:java}
if(i >= numPartitions-1)
{code}
It should be
{code:java}
if(i >= numPartitions)
{code}
Reporter: Shardul Singh
Assignee: Shardul Singh
In HRegionPartitioner getPartition(), the case where we have less reduces than
region is incorrect.
https://github.com/apache/hbase/blob/fbd5b5e32753104f88600b0f4c803ab5659bce64/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java#L92
Consider the following scenario:
1. if there are 5 regions for the table, partitions = 5 and number of reducers
is also 5.
So in this case above code should not return true. But for the last region when
i=4, getPartition should return 4 but it returns 2 because it falls in the case
of when we have less reduces than region and returns true for the above
condition.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (HBASE-22979) Call ChunkCreator.initialize is TestHRegionWithInMemoryFlush
Peter Somogyi created HBASE-22979: - Summary: Call ChunkCreator.initialize is TestHRegionWithInMemoryFlush Key: HBASE-22979 URL: https://issues.apache.org/jira/browse/HBASE-22979 Project: HBase Issue Type: Bug Reporter: Peter Somogyi Assignee: Peter Somogyi TestHRegionWithInMemoryFlush is failing 100% on branch-2.2+. Refactor of TestHRegion in HBASE-22896 did not update the overridden initHRegion method in this test. -- This message was sent by Atlassian Jira (v8.3.2#803003)
Re: [VOTE] The second HBase 2.2.1 release candidate (RC1) is available
+1 (Non-Binding) Java Version - java-1.8.0-amazon-corretto-jdk_8.222.10-1 Cluster - Fully-Distributed (3 Nodes) Hadoop Version - 3.1.2 - Built bin tarball & client tarball from Source - Checksums & Signatures - *OK* - RAT check - *OK* - ITBLL (8M) rows (& MR Tasks using Resource Manager [Verified]) - *OK* - LTT (1M) rows - *OK* - Shell [CRUD Operations] - *OK* - Web UI, Logs - *OK* - REST Server - *OK* - CRUD Operations, Web UI, Logs - *OK* - Thrift Server - *OK* - CRUD Operations, Web UI, Logs - *OK* - RS Groups - *OK* - CHANGES, RELEASENOTES & Compatibility Report - *OK* - HDFS directory structure - *OK* -Sakthi On Thu, Sep 5, 2019 at 6:35 PM Guanghao Zhang wrote: > +1 from me. > > hbase-2.2.1-bin.tar.gz (openjdk 1.8.0_202) > - Verified sha512sum: ok > - Start HBase in standalone mode: ok > - Verified with shell, create/disable/enable/drop/get/put/scan/delete: ok > - Checked master/regionserver/table/region Web UI: ok > > hbase-2.2.1-src.tar.gz (openjdk 1.8.0_202) > - Verified sha512sum: ok > - Build tarball: ok > - Start HBase in standalone mode: ok > - Verified with shell, create/disable/enable/drop/get/put/scan/delete: ok > - Checked master/regionserver/table/region Web UI: ok > > Guanghao Zhang 于2019年9月4日周三 下午5:52写道： > > > Please vote on this release candidate (RC1) for Apache HBase 2.2.1. > > > > The VOTE will remain open for at least 72 hours. > > > > [ ] +1 Release this package as Apache HBase 2.2.1 > > [ ] -1 Do not release this package because ... > > > > The tag to be voted on is 2.2.1RC1. The release files, including > > signatures, digests, etc. can be found at: > > https://dist.apache.org/repos/dist/dev/hbase/2.2.1RC1/ > > > > Maven artifacts are available in a staging repository at: > > https://repository.apache.org/content/repositories/orgapachehbase-1332/ > > > > Signatures used for HBase RCs can be found in this file: > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > > > The list of bug fixes going into 2.2.1 can be found in included > > CHANGES.md and RELEASENOTES.md available here: > > https://dist.apache.org/repos/dist/dev/hbase/2.2.1RC1/CHANGES.md > > https://dist.apache.org/repos/dist/dev/hbase/2.2.1RC1/RELEASENOTES.md > > > > A detailed source and binary compatibility report for this release is > > available at: > > > > > https://dist.apache.org/repos/dist/dev/hbase/2.2.1RC1/api_compare_2.2.1RC1_to_2.2.0.html > > > > To learn more about Apache HBase, please see http://hbase.apache.org/ > > > > Thanks, > > Guanghao Zhang > > >
