[jira] [Resolved] (HBASE-26146) Allow custom opts for hbck in hbase bin
[
https://issues.apache.org/jira/browse/HBASE-26146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-26146.
---
Fix Version/s: 2.4.6
Release Note: Adds HBASE_HBCK_OPTS environment variable to bin/hbase for
passing extra options to hbck/hbck2. Defaults to HBASE_SERVER_JAAS_OPTS if
specified, or HBASE_REGIONSERVER_OPTS.
Resolution: Fixed
Pushed #3537 to branch-2.4. Re-resolving.
Added a release note [~anoop.hbase]
> Allow custom opts for hbck in hbase bin
> ---
>
> Key: HBASE-26146
> URL: https://issues.apache.org/jira/browse/HBASE-26146
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Minor
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.4.6
>
>
> https://issues.apache.org/jira/browse/HBASE-15145 made it so that when you
> execute {{hbase hbck}}, the regionserver or JAAS opts are added automatically
> to the command line. This is problematic in some cases depending on what
> regionserver opts have been set. For instance, one might configure a jmx port
> for the regionserver but then hbck will fail due to a port conflict if run on
> the same host as a regionserver. Another example would be that a regionserver
> might define an {{-Xms}} value which is significantly more than hbck requires.
>
> We should make it possible for users to define their own HBASE_HBCK_OPTS
> which take precedence over the server opts added by default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (HBASE-26146) Allow custom opts for hbck in hbase bin
[
https://issues.apache.org/jira/browse/HBASE-26146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack reopened HBASE-26146:
---
Reopening to apply backport to branch-2.3 (#3537)
> Allow custom opts for hbck in hbase bin
> ---
>
> Key: HBASE-26146
> URL: https://issues.apache.org/jira/browse/HBASE-26146
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Minor
> Fix For: 2.5.0, 3.0.0-alpha-2
>
>
> https://issues.apache.org/jira/browse/HBASE-15145 made it so that when you
> execute {{hbase hbck}}, the regionserver or JAAS opts are added automatically
> to the command line. This is problematic in some cases depending on what
> regionserver opts have been set. For instance, one might configure a jmx port
> for the regionserver but then hbck will fail due to a port conflict if run on
> the same host as a regionserver. Another example would be that a regionserver
> might define an {{-Xms}} value which is significantly more than hbck requires.
>
> We should make it possible for users to define their own HBASE_HBCK_OPTS
> which take precedence over the server opts added by default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-26148) Backport HBASE-26146 to branch-2.4
[ https://issues.apache.org/jira/browse/HBASE-26148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-26148. --- Resolution: Invalid > Backport HBASE-26146 to branch-2.4 > -- > > Key: HBASE-26148 > URL: https://issues.apache.org/jira/browse/HBASE-26148 > Project: HBase > Issue Type: Sub-task >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Split Meta Design Reset Status
Note, no meeting tomorrow. We'll meet next week on the 4th or 5th of August. Thanks, S On Thu, Jul 22, 2021 at 8:53 PM Stack wrote: > Notes from yesterday's meeting (attendees, please amend if I misrepresent > or if you have anything extra to add!) > > Split Meta Design Reset Status > > Wed Jul 21 21:24:38 PDT 2021 > > Attendees: Bharath, Stack, Duo, and Francis > > We went over the new updates to the Brainstorming [1] section under > > Design in the Super Split Meta Design doc [2]. > > First was the new addition, 4.1.2 Extend (& Move) ConnectionRegistry; hide > > ROOT from Client [3]. In particular, filling out how "ROOT" might be > > implemented behind the new API in ConnectionRegistry. On option 1., > > replicating master-local Region to RegionServers, options considered > > included > > * Listener on master-local Region WAL to replicate. > > * Perhaps Read-Replica but master-local is not an actual Region > > * Needs to be incremental edits because ROOT could get too big to ship > >in a lump; need to visit how... > > * Possibly in-memory-only Regions on RS replicated from master-local > >Region via WAL tailing <= zhang...@apache.org > > * Which RegionServers? Those hosting ROOT replicas? > > * How to bootstrap? Failure scenarios. > > * This would be a new replication system alongside current; could > >evolve to replace/improve old? > > Duo offered to look into means of replicating the master-local Region > > out to RegionServers. > > Next up was discussion constrasting ROOT as a standalone table vs > > First-meta Region-as-Root (see 4.1.1 hbase:meta,,1 as ROOT [4]); i.e. > > options 2 and 3 for how we'd implement a ROOT. One item that came up > > was whether a need to specify one replica count for a ROOT table vs > > another for hbase:meta. If so, then it would be argument for ROOT as > > standalone table (Others of us argued it not a concern of consequence). > > If ROOT access is behind a new simple API in ConnectionRegistry, how > > to stop clients reading hbase:meta table if not Master or fronted by > > a ConnectionRegistry request? (Should be able to switch on client > > identity/source). One suggestion for First-meta-Region-as-ROOT was > > NOT returning the first Region to the client post-meta split when > > accessing via the simple API. Some concern this would confuse old > > Clients (Francis was going to take a look). > > Moved to discussion how we'd move ConnectionRegistry from > > hosted-by-Master to hosted-by-RegionServers. How to bootstrap such a > > system came up? Where do clients go? How do they know which > > RegionServers (special regionserver group? Every RS fields > > ConnectionRegistry requests but only designated core serve the ROOT > > lookup APIs?). This was a TODO. > > This led naturally into 4.1.5 System RS group for client meta services > > [5], a new addition under Brainstorming. Discussion. Bharath to look > > into feedback. > > On the end of the discussion, group expressed support for adding > > simple API to the ConnectionRegistry to hide ROOT implementation > > detail from client. Support was expressed for moving ConnectionRegistry > > from Master to RegionServers. Intent is to move forward on design of > > these pieces: e.g. how client bootstraps. > > Support was expressed for getting at least the bones of a split meta > > into an hbase3 before the RCs. > > Where we'd actually store hbase:meta Region locations -- i.e. how a > > "ROOT' would be implemented -- was for our next meeting informed by > > research of the various approaches noted mostly above. It was > > also thought that the new ConnectionRegistry should not preclude > > making progress on the "ROOT" implementation. > > Will post notice of next meeting (next Weds or the one > > following). > > 1. > https://docs.google.com/document/d/11ChsSb2LGrSzrSJz8pDCAw5IewmaMV0ZDN1LrMkAj4s/edit#heading=h.wr0fwvw06j7n > > 2. > https://docs.google.com/document/d/11ChsSb2LGrSzrSJz8pDCAw5IewmaMV0ZDN1LrMkAj4s/edit#heading=h.9s666p6no9cq > > 3. > https://docs.google.com/document/d/11ChsSb2LGrSzrSJz8pDCAw5IewmaMV0ZDN1LrMkAj4s/edit#heading=h.90th11txi153 > > 4. > https://docs.google.com/document/d/11ChsSb2LGrSzrSJz8pDCAw5IewmaMV0ZDN1LrMkAj4s/edit#heading=h.ikbhxlcthjle > > 5. > https://docs.google.com/document/d/11ChsSb2LGrSzrSJz8pDCAw5IewmaMV0ZDN1LrMkAj4s/edit#heading=h.utoenf10t05b > > On Tue, Jul 20, 2021 at 11:00 AM Stack wrote: > >> Lets meet tomorrow. Please review the design doc "Design/Brainstorming" >> Section 4.1 [1] before the meeting if you can (No harm if a refresh of the >> requirements section while you are at it). >> >> Topic: Split Meta Design Reset Status >> Time: Jul 21, 2021 05:00 PM Pacific Time (US and Canada) >> >> Join Zoom Meeting >> https://us04web.zoom.us/j/77318920525?pwd=OFZXZFVPSHJLaGNsby9SN25OV1F2Zz09 >> >> Meeting ID: 773 1892 0525 >> Passcode: hbase >> >> Thanks, >> S >> >> 1. 1. >> https://docs.google.com/document/d/11ChsSb2LGrSzrSJz8pDCAw5IewmaMV0ZDN1LrMkAj4s/edit#heading=h.w
[jira] [Created] (HBASE-26148) Backport HBASE-26146 to branch-2.4
Bryan Beaudreault created HBASE-26148: - Summary: Backport HBASE-26146 to branch-2.4 Key: HBASE-26148 URL: https://issues.apache.org/jira/browse/HBASE-26148 Project: HBase Issue Type: Sub-task Reporter: Bryan Beaudreault Assignee: Bryan Beaudreault -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [VOTE] Second release candidate for HBase 2.4.5 (RC1) is available
+1 * Signature: ok * Checksum : ok * Rat check (1.8.0_292): ok - mvn clean apache-rat:check * Built from source (1.8.0_292): ok - mvn clean install -DskipTests * Unit tests pass (1.8.0_292): ok - mvn package -P runSmallTests -Dsurefire.rerunFailingTestsCount=3 Compat report looks good. Built from src w/ release profile. Started it up in standalone mode. Loaded up data w/ pe. Verified it all made it. Poked around UI and shell. Looks good to me. S On Tue, Jul 27, 2021 at 11:09 AM Andrew Purtell wrote: > Please vote on this Apache hbase release candidate, > hbase-2.4.5RC1 > > The VOTE will remain open for at least 72 hours. > > [ ] +1 Release this package as Apache hbase 2.4.5 > [ ] -1 Do not release this package because ... > > The tag to be voted on is 2.4.5RC1: > > https://github.com/apache/hbase/tree/2.4.5RC1 > > This tag currently points to git reference 03b8c0cf42. > > The release files, including signatures, digests, as well as CHANGES.md > and RELEASENOTES.md included in this RC can be found at: > > https://dist.apache.org/repos/dist/dev/hbase/2.4.5RC1/ > > The compatibility report for this RC can be found at: > > > https://dist.apache.org/repos/dist/dev/hbase/2.4.5RC1/api_compare_2.4.4_to_2.4.5RC1.html > > There are 0 reported compatibility issues. > > Maven artifacts are available in a staging repository at: > > https://repository.apache.org/content/repositories/orgapachehbase-1462/ > > Artifacts were signed with the 0xD5365CCD key which can be found in: > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > To learn more about Apache hbase, please see > > http://hbase.apache.org/ > > Thanks, > Your HBase Release Manager >
Re: [VOTE] First release candidate for HBase 2.3.6 (RC2) is available
On Tue, Jul 27, 2021 at 4:12 PM Huaxiang Sun wrote: > Sorry, I have to put -1 on this RC. > > In release note, > https://dist.apache.org/repos/dist/dev/hbase/2.3.6RC2/CHANGES.md, > HBASE-24492 is listed in 2.3.6. > Actually, this jira is in 2.3.3, not 2.3.6. The release version in the jira > is messed up. I corrected it in the jira. > > | [HBASE-24492](https://issues.apache.org/jira/browse/HBASE-24492) | > ProtobufLogReader.readNext does not need looping | Minor | > Replication, wal | > > Ok. Let me spin another. Thanks Huaxiang. S > Huaxiang > > > On Tue, Jul 27, 2021 at 4:06 PM Nick Dimiduk wrote: > > > +1 > > > > * Compatibility report: ok > > * Nightly builds good: okay > > * Signature: ok > > * Checksum : ok > > * Rat check (1.8.0_292): ok > > - mvn clean apache-rat:check -D skipTests > > * Rat check (11.0.11): ok > > - mvn clean apache-rat:check -D skipTests -D hadoop.profile=3.0 > > * Built from source (1.8.0_292): ok > > - mvn clean install -D skipTests -DskipTests > > * Built from source (11.0.11): ok > > - mvn clean install -D skipTests -D hadoop.profile=3.0 > -DskipTests > > * Unit tests pass (1.8.0_292): ok > > - mvn package -P runAllTests -D skipTests > > -Dsurefire.rerunFailingTestsCount=3 > > * Unit tests pass (11.0.11): ok > > - mvn package -P runAllTests -D skipTests -D hadoop.profile=3.0 > > -Dsurefire.rerunFailingTestsCount=3 > > > > On Mon, Jul 26, 2021 at 11:14 PM Stack wrote: > > > > > Please vote on this Apache hbase release candidate, > > > hbase-2.3.6RC2 > > > > > > The VOTE will remain open for at least 72 hours. > > > > > > [ ] +1 Release this package as Apache hbase 2.3.6 > > > [ ] -1 Do not release this package because ... > > > > > > The tag to be voted on is 2.3.6RC2: > > > > > > https://github.com/apache/hbase/tree/2.3.6RC2 > > > > > > This tag currently points to git reference > > > > > > 79d57a453ac840873ca15fc151b6f9f342b467b9 > > > > > > The release files, including signatures, digests, as well as CHANGES.md > > > and RELEASENOTES.md included in this RC can be found at: > > > > > > https://dist.apache.org/repos/dist/dev/hbase/2.3.6RC2/ > > > > > > Maven artifacts are available in a staging repository at: > > > > > > > > https://repository.apache.org/content/repositories/orgapachehbase-1461/ > > > > > > Artifacts were signed with the 0xA7874F29 key which can be found in: > > > > > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > > > > > Known issues to be aware of: > > > > > > HBASE-26120 New replication gets stuck or data loss when multiwal > groups > > > more than 10 > > > > > > To learn more about Apache hbase, please see > > > > > > http://hbase.apache.org/ > > > > > > Thanks, > > > Your HBase Release Manager > > > > > >
Re: [VOTE] First release candidate for HBase 2.3.6 (RC2) is available
Sorry, I have to put -1 on this RC. In release note, https://dist.apache.org/repos/dist/dev/hbase/2.3.6RC2/CHANGES.md, HBASE-24492 is listed in 2.3.6. Actually, this jira is in 2.3.3, not 2.3.6. The release version in the jira is messed up. I corrected it in the jira. | [HBASE-24492](https://issues.apache.org/jira/browse/HBASE-24492) | ProtobufLogReader.readNext does not need looping | Minor | Replication, wal | Huaxiang On Tue, Jul 27, 2021 at 4:06 PM Nick Dimiduk wrote: > +1 > > * Compatibility report: ok > * Nightly builds good: okay > * Signature: ok > * Checksum : ok > * Rat check (1.8.0_292): ok > - mvn clean apache-rat:check -D skipTests > * Rat check (11.0.11): ok > - mvn clean apache-rat:check -D skipTests -D hadoop.profile=3.0 > * Built from source (1.8.0_292): ok > - mvn clean install -D skipTests -DskipTests > * Built from source (11.0.11): ok > - mvn clean install -D skipTests -D hadoop.profile=3.0 -DskipTests > * Unit tests pass (1.8.0_292): ok > - mvn package -P runAllTests -D skipTests > -Dsurefire.rerunFailingTestsCount=3 > * Unit tests pass (11.0.11): ok > - mvn package -P runAllTests -D skipTests -D hadoop.profile=3.0 > -Dsurefire.rerunFailingTestsCount=3 > > On Mon, Jul 26, 2021 at 11:14 PM Stack wrote: > > > Please vote on this Apache hbase release candidate, > > hbase-2.3.6RC2 > > > > The VOTE will remain open for at least 72 hours. > > > > [ ] +1 Release this package as Apache hbase 2.3.6 > > [ ] -1 Do not release this package because ... > > > > The tag to be voted on is 2.3.6RC2: > > > > https://github.com/apache/hbase/tree/2.3.6RC2 > > > > This tag currently points to git reference > > > > 79d57a453ac840873ca15fc151b6f9f342b467b9 > > > > The release files, including signatures, digests, as well as CHANGES.md > > and RELEASENOTES.md included in this RC can be found at: > > > > https://dist.apache.org/repos/dist/dev/hbase/2.3.6RC2/ > > > > Maven artifacts are available in a staging repository at: > > > > > https://repository.apache.org/content/repositories/orgapachehbase-1461/ > > > > Artifacts were signed with the 0xA7874F29 key which can be found in: > > > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > > > Known issues to be aware of: > > > > HBASE-26120 New replication gets stuck or data loss when multiwal groups > > more than 10 > > > > To learn more about Apache hbase, please see > > > > http://hbase.apache.org/ > > > > Thanks, > > Your HBase Release Manager > > >
Re: Rolling a 2.3.6 in next week or so (WAS => Delaying 2.3.6)
Thanks for picking this up, Stack. On Fri, Jul 23, 2021 at 9:43 AM Stack wrote: > On Thu, Jul 15, 2021 at 10:16 AM Stack wrote: > > > Get any 'critical' commits landed in next few days. Give me a shout if > you > > need help or are wondering whether a commit belongs or not. > > > > > Starting the 2.3.6 roll shout if you have a critical to get in! > S > > > > > > Thanks, > > S > > > > On Fri, Apr 30, 2021 at 9:19 AM Nick Dimiduk > wrote: > > > >> Heya, > >> > >> We only have about 25 commits on branch-2.3 since 2.3.5, which is less > >> than > >> my benchmark of at least 30 commits for a patch release. I propose we > hold > >> off another month or so. If there's a fix in that set which is critical > >> for > >> you, speak up and we can discuss it. Otherwise, let's save the PMC > >> attention for the next 2.4 release. > >> > >> Thanks, > >> Nick > >> > > >
Re: [VOTE] First release candidate for HBase 2.3.6 (RC2) is available
+1 * Compatibility report: ok * Nightly builds good: okay * Signature: ok * Checksum : ok * Rat check (1.8.0_292): ok - mvn clean apache-rat:check -D skipTests * Rat check (11.0.11): ok - mvn clean apache-rat:check -D skipTests -D hadoop.profile=3.0 * Built from source (1.8.0_292): ok - mvn clean install -D skipTests -DskipTests * Built from source (11.0.11): ok - mvn clean install -D skipTests -D hadoop.profile=3.0 -DskipTests * Unit tests pass (1.8.0_292): ok - mvn package -P runAllTests -D skipTests -Dsurefire.rerunFailingTestsCount=3 * Unit tests pass (11.0.11): ok - mvn package -P runAllTests -D skipTests -D hadoop.profile=3.0 -Dsurefire.rerunFailingTestsCount=3 On Mon, Jul 26, 2021 at 11:14 PM Stack wrote: > Please vote on this Apache hbase release candidate, > hbase-2.3.6RC2 > > The VOTE will remain open for at least 72 hours. > > [ ] +1 Release this package as Apache hbase 2.3.6 > [ ] -1 Do not release this package because ... > > The tag to be voted on is 2.3.6RC2: > > https://github.com/apache/hbase/tree/2.3.6RC2 > > This tag currently points to git reference > > 79d57a453ac840873ca15fc151b6f9f342b467b9 > > The release files, including signatures, digests, as well as CHANGES.md > and RELEASENOTES.md included in this RC can be found at: > > https://dist.apache.org/repos/dist/dev/hbase/2.3.6RC2/ > > Maven artifacts are available in a staging repository at: > > https://repository.apache.org/content/repositories/orgapachehbase-1461/ > > Artifacts were signed with the 0xA7874F29 key which can be found in: > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > Known issues to be aware of: > > HBASE-26120 New replication gets stuck or data loss when multiwal groups > more than 10 > > To learn more about Apache hbase, please see > > http://hbase.apache.org/ > > Thanks, > Your HBase Release Manager >
Re: [VOTE] First release candidate for HBase 2.3.6 (RC2) is available
(Hit send by mistake). Built src tgz and run pe and verified data landed. Poked around shell and UI. Compatibility API looks good. +1 S On Tue, Jul 27, 2021 at 3:19 PM Stack wrote: > * Signature: ok > * Checksum : ok > * Rat check (1.8.0_292): ok > - mvn clean apache-rat:check > * Built from source (1.8.0_292): ok > - mvn clean install -DskipTests > * Unit tests pass (1.8.0_292): ok > - mvn package -P runSmallTests > -Dsurefire.rerunFailingTestsCount=3 > > > > On Mon, Jul 26, 2021 at 11:14 PM Stack wrote: > >> Please vote on this Apache hbase release candidate, >> hbase-2.3.6RC2 >> >> The VOTE will remain open for at least 72 hours. >> >> [ ] +1 Release this package as Apache hbase 2.3.6 >> [ ] -1 Do not release this package because ... >> >> The tag to be voted on is 2.3.6RC2: >> >> https://github.com/apache/hbase/tree/2.3.6RC2 >> >> This tag currently points to git reference >> >> 79d57a453ac840873ca15fc151b6f9f342b467b9 >> >> The release files, including signatures, digests, as well as CHANGES.md >> and RELEASENOTES.md included in this RC can be found at: >> >> https://dist.apache.org/repos/dist/dev/hbase/2.3.6RC2/ >> >> Maven artifacts are available in a staging repository at: >> >> https://repository.apache.org/content/repositories/orgapachehbase-1461/ >> >> Artifacts were signed with the 0xA7874F29 key which can be found in: >> >> https://dist.apache.org/repos/dist/release/hbase/KEYS >> >> Known issues to be aware of: >> >> HBASE-26120 New replication gets stuck or data loss when multiwal groups >> more than 10 >> >> To learn more about Apache hbase, please see >> >> http://hbase.apache.org/ >> >> Thanks, >> Your HBase Release Manager >> >
Re: [VOTE] Second release candidate for HBase 2.4.5 (RC1) is available
+1 * Compatibility report: ok * Nightly builds good, but for occasional failed-to-read xml issue: okay * Rat check (1.8.0_292): ok - mvn clean apache-rat:check -D skipTests * Rat check (11.0.11): ok - mvn clean apache-rat:check -D skipTests -D hadoop.profile=3.0 * Built from source (1.8.0_292): ok - mvn clean install -D skipTests -DskipTests * Built from source (11.0.11): ok - mvn clean install -D skipTests -D hadoop.profile=3.0 -DskipTests * Unit tests pass (11.0.11): ok - mvn package -P runAllTests -D skipTests -D hadoop.profile=3.0 -Dsurefire.rerunFailingTestsCount=3 * Unit tests pass (1.8.0_292): ok - mvn package -P runAllTests -D skipTests -Dsurefire.rerunFailingTestsCount=3 On Tue, Jul 27, 2021 at 11:09 AM Andrew Purtell wrote: > Please vote on this Apache hbase release candidate, > hbase-2.4.5RC1 > > The VOTE will remain open for at least 72 hours. > > [ ] +1 Release this package as Apache hbase 2.4.5 > [ ] -1 Do not release this package because ... > > The tag to be voted on is 2.4.5RC1: > > https://github.com/apache/hbase/tree/2.4.5RC1 > > This tag currently points to git reference 03b8c0cf42. > > The release files, including signatures, digests, as well as CHANGES.md > and RELEASENOTES.md included in this RC can be found at: > > https://dist.apache.org/repos/dist/dev/hbase/2.4.5RC1/ > > The compatibility report for this RC can be found at: > > > https://dist.apache.org/repos/dist/dev/hbase/2.4.5RC1/api_compare_2.4.4_to_2.4.5RC1.html > > There are 0 reported compatibility issues. > > Maven artifacts are available in a staging repository at: > > https://repository.apache.org/content/repositories/orgapachehbase-1462/ > > Artifacts were signed with the 0xD5365CCD key which can be found in: > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > To learn more about Apache hbase, please see > > http://hbase.apache.org/ > > Thanks, > Your HBase Release Manager >
Re: [VOTE] First release candidate for HBase 2.3.6 (RC2) is available
* Signature: ok * Checksum : ok * Rat check (1.8.0_292): ok - mvn clean apache-rat:check * Built from source (1.8.0_292): ok - mvn clean install -DskipTests * Unit tests pass (1.8.0_292): ok - mvn package -P runSmallTests -Dsurefire.rerunFailingTestsCount=3 On Mon, Jul 26, 2021 at 11:14 PM Stack wrote: > Please vote on this Apache hbase release candidate, > hbase-2.3.6RC2 > > The VOTE will remain open for at least 72 hours. > > [ ] +1 Release this package as Apache hbase 2.3.6 > [ ] -1 Do not release this package because ... > > The tag to be voted on is 2.3.6RC2: > > https://github.com/apache/hbase/tree/2.3.6RC2 > > This tag currently points to git reference > > 79d57a453ac840873ca15fc151b6f9f342b467b9 > > The release files, including signatures, digests, as well as CHANGES.md > and RELEASENOTES.md included in this RC can be found at: > > https://dist.apache.org/repos/dist/dev/hbase/2.3.6RC2/ > > Maven artifacts are available in a staging repository at: > > https://repository.apache.org/content/repositories/orgapachehbase-1461/ > > Artifacts were signed with the 0xA7874F29 key which can be found in: > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > Known issues to be aware of: > > HBASE-26120 New replication gets stuck or data loss when multiwal groups > more than 10 > > To learn more about Apache hbase, please see > > http://hbase.apache.org/ > > Thanks, > Your HBase Release Manager >
[jira] [Created] (HBASE-26147) Add dry run mode to hbase balancer
Bryan Beaudreault created HBASE-26147: - Summary: Add dry run mode to hbase balancer Key: HBASE-26147 URL: https://issues.apache.org/jira/browse/HBASE-26147 Project: HBase Issue Type: Improvement Components: Balancer, master Reporter: Bryan Beaudreault Assignee: Bryan Beaudreault It's often rather hard to know how the cost function changes you're making will affect the balance of the cluster, and currently the only way to know is to run it. If the cost decisions are not good, you may have just moved many regions towards a non-ideal balance. Region moves themselves are not free for clients, and the resulting balance may cause a regression. We should add a mode to the balancer so that it can be invoked without actually executing any plans. This will allow an administrator to iterate on their cost functions and used the balancer's logging to see how their changes would affect the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-26146) Allow custom opts for hbck in hbase bin
[
https://issues.apache.org/jira/browse/HBASE-26146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-26146.
---
Fix Version/s: 3.0.0-alpha-2
2.5.0
Hadoop Flags: Reviewed
Resolution: Fixed
Thanks for the patch [~bbeaudreault] Merged to branch-2+. It didn't go to
branch-2.4. Conflicts. Make a subtask for a backport if you want it in 2.4/2.3
boss.
> Allow custom opts for hbck in hbase bin
> ---
>
> Key: HBASE-26146
> URL: https://issues.apache.org/jira/browse/HBASE-26146
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Minor
> Fix For: 2.5.0, 3.0.0-alpha-2
>
>
> https://issues.apache.org/jira/browse/HBASE-15145 made it so that when you
> execute {{hbase hbck}}, the regionserver or JAAS opts are added automatically
> to the command line. This is problematic in some cases depending on what
> regionserver opts have been set. For instance, one might configure a jmx port
> for the regionserver but then hbck will fail due to a port conflict if run on
> the same host as a regionserver. Another example would be that a regionserver
> might define an {{-Xms}} value which is significantly more than hbck requires.
>
> We should make it possible for users to define their own HBASE_HBCK_OPTS
> which take precedence over the server opts added by default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [VOTE] First release candidate for HBase 2.3.6 (RC2) is available
+1 (non-binding) * Signature: ok * Checksum : ok * Rat check (1.8.0_292): ok - mvn clean apache-rat:check * Built from source (1.8.0_292): ok - mvn clean install -DskipTests * Unit tests pass (1.8.0_292): ok - mvn package -P runSmallTests -Dsurefire.rerunFailingTestsCount=3 On Mon, Jul 26, 2021 at 11:15 PM Stack wrote: > > Please vote on this Apache hbase release candidate, > hbase-2.3.6RC2 > > The VOTE will remain open for at least 72 hours. > > [ ] +1 Release this package as Apache hbase 2.3.6 > [ ] -1 Do not release this package because ... > > The tag to be voted on is 2.3.6RC2: > > https://github.com/apache/hbase/tree/2.3.6RC2 > > This tag currently points to git reference > > 79d57a453ac840873ca15fc151b6f9f342b467b9 > > The release files, including signatures, digests, as well as CHANGES.md > and RELEASENOTES.md included in this RC can be found at: > > https://dist.apache.org/repos/dist/dev/hbase/2.3.6RC2/ > > Maven artifacts are available in a staging repository at: > > https://repository.apache.org/content/repositories/orgapachehbase-1461/ > > Artifacts were signed with the 0xA7874F29 key which can be found in: > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > Known issues to be aware of: > > HBASE-26120 New replication gets stuck or data loss when multiwal groups > more than 10 > > To learn more about Apache hbase, please see > > http://hbase.apache.org/ > > Thanks, > Your HBase Release Manager
Re: [VOTE] Second release candidate for HBase 2.4.5 (RC1) is available
+1 (non-binding) * Signature: ok * Checksum : ok * Rat check (1.8.0_292): ok - mvn clean apache-rat:check * Built from source (1.8.0_292): ok - mvn clean install -DskipTests * Unit tests pass (1.8.0_292): ok - mvn package -P runSmallTests -Dsurefire.rerunFailingTestsCount=3 On Tue, Jul 27, 2021 at 11:09 AM Andrew Purtell wrote: > > Please vote on this Apache hbase release candidate, > hbase-2.4.5RC1 > > The VOTE will remain open for at least 72 hours. > > [ ] +1 Release this package as Apache hbase 2.4.5 > [ ] -1 Do not release this package because ... > > The tag to be voted on is 2.4.5RC1: > > https://github.com/apache/hbase/tree/2.4.5RC1 > > This tag currently points to git reference 03b8c0cf42. > > The release files, including signatures, digests, as well as CHANGES.md > and RELEASENOTES.md included in this RC can be found at: > > https://dist.apache.org/repos/dist/dev/hbase/2.4.5RC1/ > > The compatibility report for this RC can be found at: > > > https://dist.apache.org/repos/dist/dev/hbase/2.4.5RC1/api_compare_2.4.4_to_2.4.5RC1.html > > There are 0 reported compatibility issues. > > Maven artifacts are available in a staging repository at: > > https://repository.apache.org/content/repositories/orgapachehbase-1462/ > > Artifacts were signed with the 0xD5365CCD key which can be found in: > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > To learn more about Apache hbase, please see > > http://hbase.apache.org/ > > Thanks, > Your HBase Release Manager
[VOTE] Second release candidate for HBase 2.4.5 (RC1) is available
Please vote on this Apache hbase release candidate, hbase-2.4.5RC1 The VOTE will remain open for at least 72 hours. [ ] +1 Release this package as Apache hbase 2.4.5 [ ] -1 Do not release this package because ... The tag to be voted on is 2.4.5RC1: https://github.com/apache/hbase/tree/2.4.5RC1 This tag currently points to git reference 03b8c0cf42. The release files, including signatures, digests, as well as CHANGES.md and RELEASENOTES.md included in this RC can be found at: https://dist.apache.org/repos/dist/dev/hbase/2.4.5RC1/ The compatibility report for this RC can be found at: https://dist.apache.org/repos/dist/dev/hbase/2.4.5RC1/api_compare_2.4.4_to_2.4.5RC1.html There are 0 reported compatibility issues. Maven artifacts are available in a staging repository at: https://repository.apache.org/content/repositories/orgapachehbase-1462/ Artifacts were signed with the 0xD5365CCD key which can be found in: https://dist.apache.org/repos/dist/release/hbase/KEYS To learn more about Apache hbase, please see http://hbase.apache.org/ Thanks, Your HBase Release Manager
Re: [VOTE] First release candidate for HBase 2.3.6 (RC2) is available
+1 (non-binding) * Signature: ok * Checksum : ok * Rat check (1.8.0_261): ok - mvn clean apache-rat:check -D skipTests * Built from source (1.8.0_261): ok - mvn clean install -D skipTests -DskipTests * Unit tests pass (1.8.0_261): ok - mvn package -P runAllTests -D skipTests -Dsurefire.rerunFailingTestsCount=3 A test run with 3 billion rows on a small cluster for 7 hours with chaos monkey passed. On Mon, Jul 26, 2021 at 11:14 PM Stack wrote: > Please vote on this Apache hbase release candidate, > hbase-2.3.6RC2 > > The VOTE will remain open for at least 72 hours. > > [ ] +1 Release this package as Apache hbase 2.3.6 > [ ] -1 Do not release this package because ... > > The tag to be voted on is 2.3.6RC2: > > https://github.com/apache/hbase/tree/2.3.6RC2 > > This tag currently points to git reference > > 79d57a453ac840873ca15fc151b6f9f342b467b9 > > The release files, including signatures, digests, as well as CHANGES.md > and RELEASENOTES.md included in this RC can be found at: > > https://dist.apache.org/repos/dist/dev/hbase/2.3.6RC2/ > > Maven artifacts are available in a staging repository at: > > https://repository.apache.org/content/repositories/orgapachehbase-1461/ > > Artifacts were signed with the 0xA7874F29 key which can be found in: > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > Known issues to be aware of: > > HBASE-26120 New replication gets stuck or data loss when multiwal groups > more than 10 > > To learn more about Apache hbase, please see > > http://hbase.apache.org/ > > Thanks, > Your HBase Release Manager >
[jira] [Resolved] (HBASE-26120) New replication gets stuck or data loss when multiwal groups more than 10
[
https://issues.apache.org/jira/browse/HBASE-26120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell resolved HBASE-26120.
-
Hadoop Flags: Reviewed
Resolution: Fixed
> New replication gets stuck or data loss when multiwal groups more than 10
> -
>
> Key: HBASE-26120
> URL: https://issues.apache.org/jira/browse/HBASE-26120
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 1.7.1, 2.4.5
>Reporter: Jasee Tao
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.5.0, 1.7.2, 2.3.7, 2.4.5
>
>
> {code:java}
> void preLogRoll(Path newLog) throws IOException {
> recordLog(newLog);
> String logName = newLog.getName();
> String logPrefix = DefaultWALProvider.getWALPrefixFromWALName(logName);
> synchronized (latestPaths) {
> Iterator iterator = latestPaths.iterator();
> while (iterator.hasNext()) {
> Path path = iterator.next();
> if (path.getName().contains(logPrefix)) {
> iterator.remove();
> break;
> }
> }
> this.latestPaths.add(newLog);
> }
> }
> {code}
> ReplicationSourceManager use _latestPaths_ to track each walgroup's last
> WALlog and all of them will be enqueue for replication when new replication
> peer added。
> If we set hbase.wal.regiongrouping.numgroups > 10, says 12, the name of
> WALlog group will be _regionserver.null0.timestamp_ to
> _regionserver.null11.timestamp_。*_String.contains_* is used in _preoLogRoll_
> to replace old logs in same group, leads when _regionserver.null1.ts_ comes,
> _regionserver.null11.ts_ may be replaced, and *_latestPaths_ growing with
> wrong logs*.
> Replication then partly stuckd as _regionsserver.null1.ts_ not exists on
> hdfs, and data may not be replicated to slave as _regionserver.null11.ts_ not
> in replication queue at startup.
> Because of
> [ZOOKEEPER-706|https://issues.apache.org/jira/browse/ZOOKEEPER-706], if there
> is too many logs in zk _/hbase/replication/rs/regionserver/peer_, remove_peer
> may not delete this znode, and other regionserver can't not pick up this
> queue for replication failover.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-26146) Allow custom opts for hbck in hbase bin
Bryan Beaudreault created HBASE-26146:
-
Summary: Allow custom opts for hbck in hbase bin
Key: HBASE-26146
URL: https://issues.apache.org/jira/browse/HBASE-26146
Project: HBase
Issue Type: Improvement
Reporter: Bryan Beaudreault
Assignee: Bryan Beaudreault
https://issues.apache.org/jira/browse/HBASE-15145 made it so that when you
execute {{hbase hbck}}, the regionserver or JAAS opts are added automatically
to the command line. This is problematic in some cases depending on what
regionserver opts have been set. For instance, one might configure a jmx port
for the regionserver but then hbck will fail due to a port conflict if run on
the same host as a regionserver. Another example would be that a regionserver
might define an {{-Xms}} value which is significantly more than hbck requires.
We should make it possible for users to define their own HBASE_HBCK_OPTS which
take precedence over the server opts added by default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-26145) Update fails with DoNotRetryIOException while the region is being split
Oleg Muravskiy created HBASE-26145:
--
Summary: Update fails with DoNotRetryIOException while the region
is being split
Key: HBASE-26145
URL: https://issues.apache.org/jira/browse/HBASE-26145
Project: HBase
Issue Type: Bug
Affects Versions: 2.4.4
Environment: We are running HBase compiled from the release 2.4.4 tag
with hadoop3 profile enabled, no other changes:
JVM Version Red Hat, Inc. 1.8.0_292-25.292-b10 JVM vendor and version
HBase Version 2.4.4-H3, revision=Unknown HBase version and revision
HBase Compiled Fri Jul 23 08:21:49 UTC 2021, giioper When HBase version was
compiled and by whom
HBase Source Checksum
148222e0d18db5336d86bb2fda0d12a164bcbd710cd7181a98467a5bf41fa55648d7bf626d0fc32551d62986b82a7ce214c917c0f8ab6293702610c2034660d3
HBase source SHA512 checksum
Hadoop Version 3.2.2, revision=7a3bc90b05f257c8ace2f76d74264906f0f7a932 Hadoop
version and revision
Hadoop Compiled 2021-01-03T12:14Z, hexiaoqiao When Hadoop version was compiled
and by whom
Hadoop Source Checksum 5a8f564f46624254b27f6a33126ff4 Hadoop source MD5 checksum
ZooKeeper Client Version 3.5.7,
revision=f0fdd52973d373ffd9c86b81d99842dc2c7f660e ZooKeeper client version and
revision hash
ZooKeeper Client Compiled 02/10/2020 11:30 GMT When ZooKeeper client version
was compiled
ZooKeeper Quorum zookeeper1:2181
zookeeper2:2181
zookeeper3:2181
zookeeper4:2181
zookeeper5:2181 Addresses of all registered ZK servers. For more, see zk dump.
ZooKeeper Base Path /hbase Root node of this cluster in ZK.
Cluster Key zookeeper1:2181
zookeeper2:2181
zookeeper3:2181
zookeeper4:2181
zookeeper5:2181:/hbase Key to add this cluster as a peer for replication. Use
'help "add_peer"' in the shell for details.
HBase Root Directory hdfs://dune/hbase Location of HBase home directory
HMaster Start Time Fri Jul 23 09:53:50 UTC 2021 Date stamp of when this HMaster
was started
HMaster Active Time Fri Jul 23 09:53:52 UTC 2021 Date stamp of when this
HMaster became active
HBase Cluster ID 6cd0faae-6cf3-4888-87e0-e977a45faeaf Unique identifier
generated for each HBase cluster
Load average 156.28 Average number of regions per regionserver. Naive
computation.
Coprocessors [] Coprocessors currently loaded by the master
LoadBalancer org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer
LoadBalancer to be used in the Master
Reporter: Oleg Muravskiy
While inserting records into HBase, I'm getting following exceptions on the
client side:
{noformat}
2021-07-27 12:07:33,986 WARN [hconnection-0x6d4081cd-shared-pool-158]
o.a.h.h.c.AsyncRequestFutureImpl - id=1, table=ris:ris-updates, attempt=1/16,
failureCount=9ops, last
exception=org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.ExceptionInI
nitializerError on worker008,16020,1627053306686, tracking started Tue Jul 27
12:07:33 UTC 2021; NOT retrying, failed=9 -- final attempt!
2021-07-27 12:07:34,008 ERROR [Thread-4] o.a.hadoop.hbase.client.BatchErrors -
Exception occurred! Exception details:
[org.apache.hadoop.hbase.DoNotRetryIOException:
java.lang.ExceptionInInitializerError, org.a
pache.hadoop.hbase.DoNotRetryIOException:
java.lang.ExceptionInInitializerError,
org.apache.hadoop.hbase.DoNotRetryIOException:
java.lang.ExceptionInInitializerError,
org.apache.hadoop.hbase.DoNotRetryIOException:
java.lang.ExceptionInInitializerError,
org.apache.hadoop.hbase.DoNotRetryIOException:
java.lang.ExceptionInInitializerError,
org.apache.hadoop.hbase.DoNotRetryIOException:
java.lang.ExceptionInInitializerError,
org.apache.hadoop.hbase.DoNotRetryIOException:
java.lang.ExceptionInInitializerEr
ror, org.apache.hadoop.hbase.DoNotRetryIOException:
java.lang.ExceptionInInitializerError,
org.apache.hadoop.hbase.DoNotRetryIOException:
java.lang.ExceptionInInitializerError];
Actions:
[{"totalColumns":2,"row":"\\x00\\x03\\x00\\x00\\x01z\\xE7\\xDC\\xE5z\\x04P\\xF9\\xD3\\xD9\\x00\\x00\\x00\\x00","families":{"-":[{"qualifier":"mrt","vlen":117,"tag":[],"timestamp":"1627387651450"},{"qualifier":"source","vlen":11,"tag":[],"timest
amp":"1627387651450"}]},"ts":"1627387651450"},
{"totalColumns":2,"row":"\\x00\\x03\\x00\\x00\\x01z\\xE7\\xDC\\xE5z\\x04P\\xF9\\xD1\\xB0\\x00\\x00\\x00\\x01","families":{"-":[{"qualifier":"mrt","vlen":117,"tag":[],"timestamp":"1627387651450"},{"qualifier
":"source","vlen":11,"tag":[],"timestamp":"1627387651450"}]},"ts":"1627387651450"},
{"totalColumns":2,"row":"\\x00\\x03\\x00\\x00\\x01z\\xE7\\xDC\\xE5z\\x04P\\xF9\\xD2\\xD0\\x00\\x00\\x00\\x02","families":{"-":[{"qualifier":"mrt","vlen":125,"tag":[],"ti
mestamp":"1627387651450"},{"qualifier":"source","vlen":11,"tag":[],"timestamp":"1627387651450"}]},"ts":"1627387651450"},
{"totalColumns":2,"row":"\\x00\\x03\\x00\\x00\\x01z\\xE7\\xDC\\xE5z\\x06
\\x01\\x07\\xF8\\x00\\x01\\x00\\x00\\x00\\x00\\xA5\\x02\\x9
4g\\x00\\x01\\x00\\x00\\x00\\x03","families":{"-":[{"qualifier":"mrt","vlen":175,"tag
[jira] [Created] (HBASE-26144) The HStore.snapshot method is never called in main code
Duo Zhang created HBASE-26144: - Summary: The HStore.snapshot method is never called in main code Key: HBASE-26144 URL: https://issues.apache.org/jira/browse/HBASE-26144 Project: HBase Issue Type: Improvement Components: regionserver Reporter: Duo Zhang Assignee: Duo Zhang In the comment of HStore.flushCache, we say that HStore.snapshot method must be called first. But actually, we will call memstore.snapshot directly from StoreFlusherImpl.prepare, without holding the write lock. The reason we do not need to hold the write lock is that, we hold HRegion.updatesLock in the upper layer, so it is OK. See HBASE-10087 for more discussion about this. So in general, I think we could remove the snapshot method in HStore. And I do not think we need to hold the write lock when calling clearSnapshot then. As in the HRegion layer, we need to guarantee that there is only one ongoing flush of the region, so before we finish the region, it is already safe to operate on the snapshot of the memstore, especially that the clearSnapshot method itself is thread safe. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26143) The default value of 'hbase.hregion.memstore.mslab.indexchunksize.percent' should depend on MemStore type
chenglei created HBASE-26143: Summary: The default value of 'hbase.hregion.memstore.mslab.indexchunksize.percent' should depend on MemStore type Key: HBASE-26143 URL: https://issues.apache.org/jira/browse/HBASE-26143 Project: HBase Issue Type: Bug Affects Versions: 2.4.0, 3.0.0-alpha-1 Reporter: chenglei -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26142) NullPointerException when set "hbase.hregion.memstore.mslab.indexchunksize.percent" to zero"
chenglei created HBASE-26142: Summary: NullPointerException when set "hbase.hregion.memstore.mslab.indexchunksize.percent" to zero" Key: HBASE-26142 URL: https://issues.apache.org/jira/browse/HBASE-26142 Project: HBase Issue Type: Bug Affects Versions: 2.4.0, 3.0.0-alpha-1 Reporter: chenglei -- This message was sent by Atlassian Jira (v8.3.4#803005)
