[jira] [Resolved] (HBASE-28529) Use ZKClientConfig instead of system properties when setting zookeeper configurations
[ https://issues.apache.org/jira/browse/HBASE-28529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28529. --- Fix Version/s: 2.6.0 3.0.0-beta-2 2.5.9 Hadoop Flags: Reviewed Resolution: Fixed Pushed to branch-2.5+. Thanks all for helping and reviewing! > Use ZKClientConfig instead of system properties when setting zookeeper > configurations > - > > Key: HBASE-28529 > URL: https://issues.apache.org/jira/browse/HBASE-28529 > Project: HBase > Issue Type: Improvement > Components: Zookeeper >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 3.0.0-beta-2, 2.5.9 > > > In HBASE-28340, we allow loading zookeeper configurations from hbase > configurations, but then we use system properties to pass these parameters > when creating zookeeper client. > For replication, we may want to use different zookeeper configurations > comparing to the ones we use for starting this hbase cluster, so using system > properties to pass these parameters is not suitable then. > We should make use of ZKClientConfig to pass these flags. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28255) Correcting spelling errors or annotations with non-standard spelling
[ https://issues.apache.org/jira/browse/HBASE-28255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Beaudreault resolved HBASE-28255. --- Fix Version/s: 2.6.0 3.0.0-beta-2 2.5.8 Resolution: Fixed Looks like this was forgot to resolve. I added what I think are the correct fixVersions > Correcting spelling errors or annotations with non-standard spelling > > > Key: HBASE-28255 > URL: https://issues.apache.org/jira/browse/HBASE-28255 > Project: HBase > Issue Type: Improvement >Reporter: mazhengxuan >Priority: Minor > Labels: documentation > Fix For: 2.6.0, 3.0.0-beta-2, 2.5.8 > > > Modify some spelling errors or non-standard spelling comments pointed out by > Typo -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28545) Worker thread gets stuck in an infinite retry when TRUNCATE command runs into RowTooBigException
Klay created HBASE-28545:
Summary: Worker thread gets stuck in an infinite retry when
TRUNCATE command runs into RowTooBigException
Key: HBASE-28545
URL: https://issues.apache.org/jira/browse/HBASE-28545
Project: HBase
Issue Type: Bug
Affects Versions: 2.5.8, 2.4.17
Reporter: Klay
Attachments: hbase-master.log
Hi, I am testing the reliability of HBase to see if it could work normally or
properly handle exceptions under certain different configurations.
When setting +hbase.table.max.rowsize+ to a relatively small size, create an
empty table and then truncate it could get worker thread to get stuck in an
infinite retry and generate massive logs.
h1. Reproduce
This bug can be reproduced determinstically with the following steps:
1. Set hbase.table.max.rowsize to a relatively small value: 745
hbase-site.xml
{code:java}
hbase.table.max.rowsize
745
{code}
2. Start up HBase cluster 2.5.8 (1HM, 2RS) and execute the following commands
from hbase shell
{code:java}
create 'uuid0f06811e5d11431fa9d4a543705b8747', {NAME =>
'uuidad7c09870a6346598bbca56cb97c4c67', VERSIONS => 4, COMPRESSION => 'GZ',
BLOOMFILTER => 'NONE', IN_MEMORY => 'false'}, {NAME =>
'uuidd2c5ad4df7b844afb4c2ac92b224dedd', VERSIONS => 4, COMPRESSION => 'NONE',
BLOOMFILTER => 'NONE', IN_MEMORY => 'true'}, {NAME =>
'uuid6e21f5f492eb4b9ba0e431b9a0666dae', VERSIONS => 4, COMPRESSION => 'NONE',
BLOOMFILTER => 'NONE', IN_MEMORY => 'true'}, {NAME =>
'uuidfb110629fba44915956e41609b68ebe9', VERSIONS => 2, COMPRESSION => 'NONE',
BLOOMFILTER => 'ROW', IN_MEMORY => 'false'}, {NAME =>
'uuid4943640bf1f6425582ac9b3572d95964', VERSIONS => 4, COMPRESSION => 'GZ',
BLOOMFILTER => 'ROWCOL', IN_MEMORY => 'false'}
truncate 'uuid0f06811e5d11431fa9d4a543705b8747' {code}
*truncate* command timeout after 10m. However, the PEWorker thread still gets
stuck in an *infinite retry* and keep generating a massive WARN in HMaster logs
{code:java}
➜ log git:(main) ✗ ls
hbase--master-09e0b9e25e7c.log hbase--master-09e0b9e25e7c.log.12
hbase--master-09e0b9e25e7c.log.16 hbase--master-09e0b9e25e7c.log.2
hbase--master-09e0b9e25e7c.log.5 hbase--master-09e0b9e25e7c.log.9
SecurityAuth.audit
hbase--master-09e0b9e25e7c.log.1 hbase--master-09e0b9e25e7c.log.13
hbase--master-09e0b9e25e7c.log.17 hbase--master-09e0b9e25e7c.log.20
hbase--master-09e0b9e25e7c.log.6 hbase--master-09e0b9e25e7c.out
hbase--master-09e0b9e25e7c.log.10 hbase--master-09e0b9e25e7c.log.14
hbase--master-09e0b9e25e7c.log.18 hbase--master-09e0b9e25e7c.log.3
hbase--master-09e0b9e25e7c.log.7 hbase--zookeeper-09e0b9e25e7c.log
hbase--master-09e0b9e25e7c.log.11 hbase--master-09e0b9e25e7c.log.15
hbase--master-09e0b9e25e7c.log.19 hbase--master-09e0b9e25e7c.log.4
hbase--master-09e0b9e25e7c.log.8 hbase--zookeeper-09e0b9e25e7c.out {code}
The logs contain the following exceptions
{code:java}
2024-04-23T02:52:16,144 WARN [PEWorker-14] procedure.TruncateTableProcedure:
Retriable error trying to truncate table=uuid0f06811e5d11431fa9d4a543705b8747
state=TRUNCATE_TABLE_CLEAR_FS_LAYOUT
org.apache.hadoop.hbase.regionserver.RowTooBigException:
org.apache.hadoop.hbase.regionserver.RowTooBigException: Max row size allowed:
745, but the row is bigger than that, the row info:
uuid0f06811e5d11431fa9d4a543705b8747,,1713840709844.c03450373b32cba95a2dd3139f0a2201./info:state/1713840716052/Put/vlen=6/seqid=16,
already have process row cells = 6, it belong to region =
hbase:meta,,1.1588230740
at
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:653)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:145)
at
org.apache.hadoop.hbase.regionserver.RegionScannerImpl.populateResult(RegionScannerImpl.java:342)
at
org.apache.hadoop.hbase.regionserver.RegionScannerImpl.nextInternal(RegionScannerImpl.java:513)
at
org.apache.hadoop.hbase.regionserver.RegionScannerImpl.nextRaw(RegionScannerImpl.java:278)
at
org.apache.hadoop.hbase.regionserver.RegionScannerImpl.next(RegionScannerImpl.java:256)
at
org.apache.hadoop.hbase.regionserver.RegionScannerImpl.next(RegionScannerImpl.java:243)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2654)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2577)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:45002)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:415)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)
at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82)
at sun.reflect.GeneratedConstructorAcces
[jira] [Created] (HBASE-28546) Make WAL rolling exception clear
Wei-Chiu Chuang created HBASE-28546:
---
Summary: Make WAL rolling exception clear
Key: HBASE-28546
URL: https://issues.apache.org/jira/browse/HBASE-28546
Project: HBase
Issue Type: Bug
Reporter: Wei-Chiu Chuang
Assignee: Wei-Chiu Chuang
Occasionally we see errors like this that doesn't really give much clue what
went wrong.
{noformat}
2024-04-22 08:08:02,026 ERROR org.apache.hadoop.hbase.master.HMaster: *
ABORTING master ccycloud-7.ozn-hb973chf3oz.xyz,22001,1713770648404: Log rolling
failed *
java.lang.RuntimeException
at
org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:217)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(AsyncProtobufLogWriter.java:223)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:164)
at
org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:116)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:726)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:129)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:886)
at
org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(AbstractWALRoller.java:304)
at
org.apache.hadoop.hbase.wal.AbstractWALRoller.run(AbstractWALRoller.java:211)
{noformat}
In this case, it was due to a time out exception. It would be helpful to make
the log message more friendly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-28436) Use connection url to specify the connection registry information
[ https://issues.apache.org/jira/browse/HBASE-28436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28436. --- Fix Version/s: 2.7.0 3.0.0-beta-2 Hadoop Flags: Reviewed Resolution: Fixed Merged to branch-2+. Thanks all for reviewing and helping! Will fill the release note soon. > Use connection url to specify the connection registry information > - > > Key: HBASE-28436 > URL: https://issues.apache.org/jira/browse/HBASE-28436 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 2.7.0, 3.0.0-beta-2 > > > As describe in this email from [~ndimiduk] > https://lists.apache.org/thread/98wqlkqvlnmpx3r7yrg9mw4pqz9ppofh > The first advantage here is that, we can encode the connection registry > implementation in the scheme of the connection url, so for replication, we > can now support cluster key other than zookeeper, which is important for us > to remove zookeeper dependency on our public facing APIs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28547) Support specifying connection registry configuration through queries of the connection uri
Duo Zhang created HBASE-28547: - Summary: Support specifying connection registry configuration through queries of the connection uri Key: HBASE-28547 URL: https://issues.apache.org/jira/browse/HBASE-28547 Project: HBase Issue Type: Sub-task Components: Client Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28548) Add documentation about the URI based connection uri
Duo Zhang created HBASE-28548: - Summary: Add documentation about the URI based connection uri Key: HBASE-28548 URL: https://issues.apache.org/jira/browse/HBASE-28548 Project: HBase Issue Type: Sub-task Components: Client, documentation Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28549) Make shell commands support column qualifiers with colons
Junegunn Choi created HBASE-28549:
-
Summary: Make shell commands support column qualifiers with colons
Key: HBASE-28549
URL: https://issues.apache.org/jira/browse/HBASE-28549
Project: HBase
Issue Type: Bug
Components: shell
Reporter: Junegunn Choi
Revisiting abandonded HBASE-13788.
h2. Problem
Shell commands do not support column qualifiers with colons (which are actually
quite common in practice) because the part after the first colon is always
treated as a converter expression.
This can be too restrictive because:
# Converters are only used for {{get}} and {{scan}} commands. They are not
supported anyway for other mutating commands such as {{put}}, {{delete}},
{{incr}}, etc.
# Converter expression should follow a specific pattern. It should either be a
method name of the {{Bytes}} class, or should be in {{c(CLASS).METHOD}} format.
We ignore the part after the first colon even if it does not follow the pattern.
h2. Suggested solution
I suggest applying two approaches to make the commands support column
qualifiers with colons.
# Do not interpret column qualifiers when using commands that don't support
converters.
# If the part after the first colon does not follow the pattern, treat it as a
part of the column qualifier
h2. Counterargument
Depending on how you see it, this makes the commands inconsistent in how they
handle column qualifiers. For example, a user may want to use the same column
expression throughout the commands.
{code}
create 't1', 'cf'
col = 'cf:cq:toLong'
# Expecting incr command to automatically ignore :toLong part
incr 't1', 'r1', col, 1
get 't1', 'r1', COLUMNS => [col]
{code}
However, if we see the converter as an option that is supported by only a few
commands, passing it to a command that doesn't support it can be considered to
be a user error. {{help 'put'}} or {{help 'delete'}} don't mention anything
about converters.
h2. Alternative solution
An alternative solution would be to add a global switch that disables the
converter interpretation altogether.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
