[jira] [Created] (HBASE-24191) HRegion#processRowsWithLocks count memstore size wrong sync failed.
Lijin Bin created HBASE-24191: - Summary: HRegion#processRowsWithLocks count memstore size wrong sync failed. Key: HBASE-24191 URL: https://issues.apache.org/jira/browse/HBASE-24191 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 1.7.0 Reporter: Lijin Bin Assignee: Lijin Bin HRegion#processRowsWithLocks when wal sync failed, it will roll back the memstore in Store and reduce the size in DefaultMemStore, but still add memStoreSize to HRegion#memstoreSize. And this only affect branch-1 verions, not apply to master/branch-2. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24190) Case-sensitive use of configuration parameter hbase.security.authentication
zhangbuzhang created HBASE-24190:
Summary: Case-sensitive use of configuration parameter
hbase.security.authentication
Key: HBASE-24190
URL: https://issues.apache.org/jira/browse/HBASE-24190
Project: HBase
Issue Type: Bug
Components: conf
Reporter: zhangbuzhang
In hbase-20586 (https://issues.apache.org/jira/browse/HBASE-20586)
（commit_sha: [https://github.com/apache/hbase/commit/cd61bcc0] ）
The code added
([SyncTable.java|https://github.com/apache/hbase/commit/cd61bcc0#diff-d1b79635f33483bf6226609e91fd1cc3])
for the use of *hbase.security.authentication* is case-sensitive. So users
setting it to “KERBEROS” won’t take effect.
private void initCredentialsForHBase(String zookeeper, Job job) throws
IOException {
Configuration peerConf = HBaseConfiguration.createClusterConf(job
.getConfiguration(), zookeeper);
if(peerConf.get("hbase.security.authentication").equals("kerberos")){
TableMapReduceUtil.initCredentialsForCluster(job, peerConf);
}
}
However, in current code base, other uses of *hbase.security.authentication*
are all case-insensitive. For example in *MasterFileSystem.java.*
public MasterFileSystem(Configuration conf) throws IOException {
...
this.isSecurityEnabled =
"kerberos".equalsIgnoreCase(conf.get("hbase.security.authentication"));
...
}
Users may misconfigured the parameter because of the case-senstive problem.
*How To Fix*
Using *eqaulsIgnoreCase* API consistently in every place when using
*hbase.security.authentication* or make it clear in Doc. (Current doc doesn't
mention it clearly)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-23994) Add WebUI to Canary
[
https://issues.apache.org/jira/browse/HBASE-23994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-23994.
--
Fix Version/s: 2.3.0
3.0.0
Resolution: Fixed
Applied to branch-2.3+. [~GeorryHuang] if you'd like to prepare a patch for
branch-1, we can back port it there as well. Thanks for the nice contribution!
> Add WebUI to Canary
>
>
> Key: HBASE-23994
> URL: https://issues.apache.org/jira/browse/HBASE-23994
> Project: HBase
> Issue Type: Improvement
> Components: canary, website
>Affects Versions: 3.0.0
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0
>
> Attachments: image-2020-03-16-09-12-00-595.png
>
>
> During the running of Canary, the table sniff failure information will be
> printed through the Log.
> {code:java}
> LOG.error("Read from {} on {}", table, server);
> {code}
>
> I think we can use WebUI to display these failures to make it easier for us
> to view this information
>
> !image-2020-03-16-09-12-00-595.png!
> As shown in the figure above, we can directly see the Table and Regionserver
> where the error occurred
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24189) Regionserver recreates region folders in HDFS after replaying WAL with removed table entries
Andrey Elenskiy created HBASE-24189: --- Summary: Regionserver recreates region folders in HDFS after replaying WAL with removed table entries Key: HBASE-24189 URL: https://issues.apache.org/jira/browse/HBASE-24189 Project: HBase Issue Type: Bug Components: regionserver, wal Affects Versions: 2.2.4 Environment: * HDFS 3.1.3 * HBase 2.1.4 * OpenJDK 8 Reporter: Andrey Elenskiy Under the following scenario region directories in HDFS can be recreated with only recovered.edits in them: # Create table "test" # Put into "test" # Delete table "test" # Create table "test" again # Crash the regionserver to which the put has went to force the WAL replay # Region directory in old table is recreated in new table # hbase hbck returns inconsistency This appears to happen due to the fact that WALs are not cleaned up once a table is deleted and they still contain the edits from old table. I've tried wal_roll command on the regionserver before crashing it, but it doesn't seem to help as under some circumstances there are still WAL files around. The only solution that works consistently is to restart regionserver before creating the table at step 4 because that triggers log cleanup on startup: [https://github.com/apache/hbase/blob/f3ee9b8aa37dd30d34ff54cd39fb9b4b6d22e683/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java#L508|https://github.com/apache/hbase/blob/f3ee9b8aa37dd30d34ff54cd39fb9b4b6d22e683/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java#L508)] Truncating a table also would be a workaround by in our case it's a no-go as we create and delete tables in our tests which run back to back (create table in the beginning of the test and delete in the end of the test). A nice option in our case would be to provide hbase shell utility to force clean up of log files manually as I realize that it's not really viable to clean all of those up every time some table is removed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24124) hbase-filesystem to use guava from hbase-thirdparty
[
https://issues.apache.org/jira/browse/HBASE-24124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang resolved HBASE-24124.
-
Resolution: Fixed
Thanks the review from [~tamaas] and [~busbey]!
> hbase-filesystem to use guava from hbase-thirdparty
> ---
>
> Key: HBASE-24124
> URL: https://issues.apache.org/jira/browse/HBASE-24124
> Project: HBase
> Issue Type: Task
> Components: Filesystem Integration
>Affects Versions: 1.0.0-alpha1
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Fix For: 1.0.0-alpha2
>
>
> hbase-filesystem repo is on guava23.0:
> {noformat}
> $ grep -r "guava" .
> ./pom.xml:23.0
> ./hbase-oss/pom.xml: com.google.guava
> ./hbase-oss/pom.xml: guava
> ./hbase-oss/pom.xml: ${guava.version}
> ./hbase-oss/pom.xml:
[jira] [Resolved] (HBASE-24180) Edit test doc around forkcount and speeding up test runs
[ https://issues.apache.org/jira/browse/HBASE-24180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-24180. --- Fix Version/s: 3.0.0 Hadoop Flags: Reviewed Resolution: Fixed Pushed on master branch. Thanks for review [~janh] > Edit test doc around forkcount and speeding up test runs > > > Key: HBASE-24180 > URL: https://issues.apache.org/jira/browse/HBASE-24180 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0 > > > Give the 'Running unit tests' section an edit. Fold in findings of late > around how to speed up tests, hardware limits and sizings, etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24188) PR Hanger. Ignore
Michael Stack created HBASE-24188: - Summary: PR Hanger. Ignore Key: HBASE-24188 URL: https://issues.apache.org/jira/browse/HBASE-24188 Project: HBase Issue Type: Bug Reporter: Michael Stack -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24187) Document usage of hbase.regionserver.fair.region.close.lock
Viraj Jasani created HBASE-24187: Summary: Document usage of hbase.regionserver.fair.region.close.lock Key: HBASE-24187 URL: https://issues.apache.org/jira/browse/HBASE-24187 Project: HBase Issue Type: Task Reporter: Viraj Jasani -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24186) RegionMover ignores replicationId
Szabolcs Bukros created HBASE-24186: --- Summary: RegionMover ignores replicationId Key: HBASE-24186 URL: https://issues.apache.org/jira/browse/HBASE-24186 Project: HBase Issue Type: Bug Components: read replicas Affects Versions: master Reporter: Szabolcs Bukros Assignee: Szabolcs Bukros When RegionMover looks up which rs hosts a region, it does this based on startRowKey. When read replication is enabled this might not return the expected region's data and this can prevent the moving of these regions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-24072) Nightlies reporting OutOfMemoryError: unable to create new native thread
[
https://issues.apache.org/jira/browse/HBASE-24072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack reopened HBASE-24072:
---
Reopening. Just as I closed this because we hadn't seen this failure in tests
in a while, last night branch-2.3 failed with this.
The ulimit-a shows that the host had 3 as its ulimit -u. The checkout had
HBASE-24126 Up the container nproc uplimit from 1 to 12500 (#1504) in it
too.
Reopening to see why on branch-2.3 we got this and to figure if general problem
still.
> Nightlies reporting OutOfMemoryError: unable to create new native thread
>
>
> Key: HBASE-24072
> URL: https://issues.apache.org/jira/browse/HBASE-24072
> Project: HBase
> Issue Type: Task
> Components: test
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-24072-Nightlies-reporting-OutOfMemoryError-una.patch,
> print_ulimit.patch
>
>
> Seeing this kind of thing in nightly...
> {code}
> java.lang.RuntimeException: java.lang.OutOfMemoryError: unable to create new
> native thread
> at
> org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper.beforeClass(TestMultithreadedTableMapper.java:83)
> Caused by: java.lang.OutOfMemoryError: unable to create new native thread
> at
> org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper.beforeClass(TestMultithreadedTableMapper.java:83)
> {code}
> Chatting w/ Nick and Huaxiang, doing the math, we are likely oversubscribing
> our docker container. It is set to 20G (The hosts are 48G). Fork count is
> 0.5C on a 16 CPU machine which is 8 *2.8G our current forked jvm size. Add
> the maven 4G and we could be over the top.
> Play w/ downing the fork size (in earlier study we didn't seem to need this
> much RAM when running a fat long test). Let me also take th ms off the mvn
> allocation to see if that helps.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24185) Junit tests do not behave well with System.exit or Runtime.halt or JVM exits in general.
Mark Robert Miller created HBASE-24185: -- Summary: Junit tests do not behave well with System.exit or Runtime.halt or JVM exits in general. Key: HBASE-24185 URL: https://issues.apache.org/jira/browse/HBASE-24185 Project: HBase Issue Type: Test Components: test Reporter: Mark Robert Miller This ends up exiting the JVM and confusing / erroring out the test runner that manages that JVM as well as cutting off test output files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24181) Add region info when log meessages in HRegion.
[ https://issues.apache.org/jira/browse/HBASE-24181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lijin Bin resolved HBASE-24181. --- Resolution: Fixed > Add region info when log meessages in HRegion. > -- > > Key: HBASE-24181 > URL: https://issues.apache.org/jira/browse/HBASE-24181 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 3.0.0 >Reporter: Lijin Bin >Assignee: Lijin Bin >Priority: Minor > Fix For: 3.0.0, 2.3.0, 2.2.5 > > > Some log message do not have region info when log, need to add it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24184) listSnapshots returns empty when just use simple acl but not use authentication
tianhang tang created HBASE-24184: - Summary: listSnapshots returns empty when just use simple acl but not use authentication Key: HBASE-24184 URL: https://issues.apache.org/jira/browse/HBASE-24184 Project: HBase Issue Type: Bug Components: snapshots Reporter: tianhang tang Assignee: tianhang tang For the owner of snapshots(not global admin user), currently list_snapshots returns empty if i just use simple acls for authorization but not use authentication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24183) [flakey test] replication.TestAddToSerialReplicationPeer
Huaxiang Sun created HBASE-24183:
Summary: [flakey test] replication.TestAddToSerialReplicationPeer
Key: HBASE-24183
URL: https://issues.apache.org/jira/browse/HBASE-24183
Project: HBase
Issue Type: Test
Components: Client
Affects Versions: 3.0.0, 2.3.0, 2.4.0
Reporter: Huaxiang Sun
Assignee: Hua Xiang
Fix For: 3.0.0, 2.3.0
>From 2.3 flakey board,
>[https://builds.apache.org/job/HBase-Flaky-Tests/job/branch-2.3/199/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.locking.TestEntityLocks.txt]
{code:java}
---
Test set: org.apache.hadoop.hbase.client.locking.TestEntityLocks
---
Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 3.917 s <<<
FAILURE! - in org.apache.hadoop.hbase.client.locking.TestEntityLocks
org.apache.hadoop.hbase.client.locking.TestEntityLocks.testEntityLockTimeout
Time elapsed: 2.022 s <<< FAILURE!
java.lang.AssertionError
at
org.apache.hadoop.hbase.client.locking.TestEntityLocks.testEntityLockTimeout(TestEntityLocks.java:178)
Mapping to the code, it complains
assertFalse(lock.getWorker().isAlive());
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
