[jira] [Created] (HBASE-27476) Recovered replication may be blocked if enabled hbase.separate.oldlogdir.by.regionserver
Sun Xin created HBASE-27476:
---
Summary: Recovered replication may be blocked if enabled
hbase.separate.oldlogdir.by.regionserver
Key: HBASE-27476
URL: https://issues.apache.org/jira/browse/HBASE-27476
Project: HBase
Issue Type: Bug
Components: Replication
Affects Versions: 2.4.15, 3.0.0-alpha-3
Reporter: Sun Xin
Assignee: Sun Xin
In other PR, I got a failed UT
{code:java}
[ERROR] Failures:
[ERROR]
org.apache.hadoop.hbase.replication.TestReplicationKillMasterRSWithSeparateOldWALs.killOneMasterRS
[ERROR] Run 1:
TestReplicationKillMasterRSWithSeparateOldWALs>TestReplicationKillMasterRS.killOneMasterRS:47->TestReplicationKillRS.loadTableAndKillRS:84
Waited too much time for queueFailover replication. Waited 61065ms.
[ERROR] Run 2:
TestReplicationKillMasterRSWithSeparateOldWALs>TestReplicationKillMasterRS.killOneMasterRS:47->TestReplicationKillRS.loadTableAndKillRS:84
Waited too much time for queueFailover replication. Waited 58864ms.
[ERROR] Run 3:
TestReplicationKillMasterRSWithSeparateOldWALs>TestReplicationKillMasterRS.killOneMasterRS:47->TestReplicationKillRS.loadTableAndKillRS:84
Waited too much time for queueFailover replication. Waited 57103ms. {code}
This should be caused by a bug.
If enabled {_}hbase.separate.oldlogdir.by.regionserver{_}, old wals will be
moved into different dir by regionserver name like root/oldWALs/server1/wal1 .
For recovered replication, can't convert wal path(like root/oldWALs/wal1) into
such paths, and throws FileNotFoundException.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-27475) Use different jdks when running hadoopcheck in personality scripts
Duo Zhang created HBASE-27475: - Summary: Use different jdks when running hadoopcheck in personality scripts Key: HBASE-27475 URL: https://issues.apache.org/jira/browse/HBASE-27475 Project: HBase Issue Type: Sub-task Components: jenkins, scripts Reporter: Duo Zhang Assignee: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27474) Evict blocks on split/merge; Avoid caching reference/hlinks if compaction is enabled
Wellington Chevreuil created HBASE-27474: Summary: Evict blocks on split/merge; Avoid caching reference/hlinks if compaction is enabled Key: HBASE-27474 URL: https://issues.apache.org/jira/browse/HBASE-27474 Project: HBase Issue Type: Improvement Affects Versions: 3.0.0-alpha-3 Reporter: Wellington Chevreuil Assignee: Wellington Chevreuil This change aims to improve block cache usage upon splits/merges. On a split/merge event the following main steps happen: 1) parent regions are closed; 2) daughters are created and opened with refs/hlinks; 3) Compaction is triggered soon after the daughters get online; With "hbase.rs.evictblocksonclose" set to false, we keep all blocks for the closed regions in 1, then will try to load same blocks again on 2 (since we are using the refs/links for the cache key), just to throw it away and cache the compaction resulting file in 3. If the block cache is close to its capacity, blocks from the compacted files in 3 will likely miss the cache. The proposal here is to always evict blocks for parent regions on a split/merge event, and also avoid caching blocks for refs/hlinks if compactions are enabled. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27473) Fix spotbugs warnings in hbase-rest Client.getResponseBody
Duo Zhang created HBASE-27473: - Summary: Fix spotbugs warnings in hbase-rest Client.getResponseBody Key: HBASE-27473 URL: https://issues.apache.org/jira/browse/HBASE-27473 Project: HBase Issue Type: Bug Components: REST Reporter: Duo Zhang Assignee: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-18585) Update release instructions involving gpg-agent
[
https://issues.apache.org/jira/browse/HBASE-18585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-18585.
--
Resolution: Won't Fix
The release process is now automated via the `do-release` family of scripts.
Instead of manual intervention and documentation, we can now rely on updates
those scripts to handle any such remaining edge cases.
> Update release instructions involving gpg-agent
> ---
>
> Key: HBASE-18585
> URL: https://issues.apache.org/jira/browse/HBASE-18585
> Project: HBase
> Issue Type: Task
> Components: build, documentation
>Reporter: Nick Dimiduk
>Priority: Minor
>
> Could be what I ran into is hyper-local, but making a note here none the
> less. For the build of 1.1.12RC0, I stumbled into MGPG-59. Presumably this is
> because of an graded version of {{gpg-agent}}. The release build takes some
> time, such that even if you purse the advice of invoking gpg before starting
> the build, the default config expires before the build finished on my
> machine. I haven't tested this out yet, but a solution appears to look
> something like
> # configure {{gpg-agent}} with a longer value for {{default-cache-ttl}} in
> {{$HOME/.gnupg/gpg-agent.conf}}.
> # invoke {{gpg}} such that the agent will cache the passphrase. MGPG-59
> suggests signing some random file.
> # run {{./dev-support/make_release.sh}} as normal.
> My context:
> {noformat}
> diocles:~ ndimiduk$ sw_vers
> ProductName:Mac OS X
> ProductVersion: 10.11.6
> BuildVersion: 15G1611
> diocles:~ ndimiduk$ gpg-agent --version
> gpg-agent (GnuPG) 2.1.23
> libgcrypt 1.8.0
> Copyright (C) 2017 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [DISCUSS] Use java11 for general checks in pre commit and nightly
Oh, there is a problem for branch-2.x as we still need to support hadoop2, but in our pom, when building with java 11, we add an enforcer rule to force using hadoop-3.0 profile. I think for hadoop 2.10.2, it is OK to compile and run with jdk11 since we do not need to compile hadoop itself. Let me at least run all the UTs with java 11 to see if there are any java 11 specific problems. Thanks. 张铎(Duo Zhang) 于2022年11月7日周一 18:14写道： > > Will merge the PR today if there are no big concerns. > > Thanks. > > 张铎(Duo Zhang) 于2022年11月1日周二 23:14写道： > > > > I've put up a PR for HBASE-27443, to change the version of the jdk for > > our general checks to 11. > > > > https://github.com/apache/hbase/pull/4845 > > > > This could be good progress on moving up to jdk11, as we have > > discussed dropping jdk8 support for maybe HBase 4.0. And we could also > > bump the version of error prone plugin as starting from 2.11.0, error > > prone plugin only supports jdk11+. > > > > PTAL if you have interest. > > > > Thanks.
[jira] [Created] (HBASE-27472) The personality script set wrong hadoop2 check version for branch-2
Duo Zhang created HBASE-27472: - Summary: The personality script set wrong hadoop2 check version for branch-2 Key: HBASE-27472 URL: https://issues.apache.org/jira/browse/HBASE-27472 Project: HBase Issue Type: Bug Components: jenkins, scripts Reporter: Duo Zhang We missed a condition while changing the supported hadoop version, which made our branch-2 hadoopcheck skip testing any hadoop2 versions... -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (HBASE-27443) Use java11 in the general check of our jenkins job
[ https://issues.apache.org/jira/browse/HBASE-27443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-27443: --- Found a problem that. on branch-2.x, we still need to test compile against hadoop2. but there is an enforcer rule to force using hadoop 3.0 profile when java 11 is used... Let me think how to deal with this. Typiecally, since we are using --release 8 now, there is no differences between using jdk8 or jdk11 when compiling... > Use java11 in the general check of our jenkins job > -- > > Key: HBASE-27443 > URL: https://issues.apache.org/jira/browse/HBASE-27443 > Project: HBase > Issue Type: Task > Components: build, jenkins >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.2, 2.4.16 > > > In this way we could also bump the version of error prone plugin. -- This message was sent by Atlassian Jira (v8.20.10#820010)
