[jira] [Created] (HBASE-27371) Bump spotbugs version
Duo Zhang created HBASE-27371: - Summary: Bump spotbugs version Key: HBASE-27371 URL: https://issues.apache.org/jira/browse/HBASE-27371 Project: HBase Issue Type: Task Components: build, pom Reporter: Duo Zhang Assignee: Duo Zhang Recently there are strange spotbugs failures in pre commit builds, likely to be a false positive. https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4779/2/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html Let's see if upgrading the spotbugs version can solve the problem. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27370) Avoid decompressing blocks when reading from bucket cache prefetch threads
Wellington Chevreuil created HBASE-27370:
Summary: Avoid decompressing blocks when reading from bucket cache
prefetch threads
Key: HBASE-27370
URL: https://issues.apache.org/jira/browse/HBASE-27370
Project: HBase
Issue Type: Improvement
Reporter: Wellington Chevreuil
Assignee: Wellington Chevreuil
When prefetching blocks into bucket cache, we had observed a consistent CPU
usage around 70% with no other workloads ongoing. For large bucket caches (i.e.
when using file based bucket cache), the prefetch can last for sometime and
having such a high CPU usage may impact the database usage by client
applications.
Further analysis of the prefetch threads stack trace showed that very often,
decompress logic is being executed by these threads:
{noformat}
"hfile-prefetch-1654895061122" #234 daemon prio=5 os_prio=0
tid=0x557bb2907000 nid=0x406d runnable [0x7f294a504000]
java.lang.Thread.State: RUNNABLE
at
org.apache.hadoop.io.compress.snappy.SnappyDecompressor.decompressBytesDirect(Native
Method)
at
org.apache.hadoop.io.compress.snappy.SnappyDecompressor.decompress(SnappyDecompressor.java:235)
at
org.apache.hadoop.io.compress.BlockDecompressorStream.decompress(BlockDecompressorStream.java:88)
at
org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:105)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
- locked <0x0002d24c0ae8> (a java.io.BufferedInputStream)
at
org.apache.hadoop.hbase.io.util.BlockIOUtils.readFullyWithHeapBuffer(BlockIOUtils.java:105)
at
org.apache.hadoop.hbase.io.compress.Compression.decompress(Compression.java:465)
at
org.apache.hadoop.hbase.io.encoding.HFileBlockDefaultDecodingContext.prepareDecoding(HFileBlockDefaultDecodingContext.java:90)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock.unpack(HFileBlock.java:650)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderImpl.readBlock(HFileReaderImpl.java:1342)
{noformat}
This is because *HFileReaderImpl.readBlock* is always decompressing blocks even
when *hbase.block.data.cachecompressed* is set to true.
This patch proposes an alternative flag to differentiate prefetch from normal
reads, so that doesn't decompress DATA blocks when prefetching with
*hbase.block.data.cachecompressed* set to true.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-27369) BufferedMutator#mutate method supports mutation type verification
LiangJun He created HBASE-27369: --- Summary: BufferedMutator#mutate method supports mutation type verification Key: HBASE-27369 URL: https://issues.apache.org/jira/browse/HBASE-27369 Project: HBase Issue Type: Bug Components: Client Affects Versions: 3.0.0-alpha-4 Reporter: LiangJun He Assignee: LiangJun He At present, BufferedMutator#mutate method only tell the user in the comment that it only supports Put / Delete. But it is not verified in the code implementation, so we need to deal with it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27368) Do not need to throw IllegalStateException when peer is not active in ReplicationSource.initialize
Duo Zhang created HBASE-27368: - Summary: Do not need to throw IllegalStateException when peer is not active in ReplicationSource.initialize Key: HBASE-27368 URL: https://issues.apache.org/jira/browse/HBASE-27368 Project: HBase Issue Type: Bug Components: regionserver, Replication Reporter: Duo Zhang Assignee: Duo Zhang The only place where we can reset the sourceRunning flag to false is in terminate method, where we want to stop the replication source. And we already have a interrupted state check in the initialize method to not throw IllegalStateException when peer is terminating, so I think here we could just do not throw IllegalStateExeption at all. The reason why the interrupted state check does not work is that, in FutureUtils.get, we will wrap InterruptedException with InterruptedIOException, without restoring the interrupted state, but the upper layer does not treat InterruptedIOException specially so the interrupted state is lost. But anway, I do not think we should rely on the interrupted state to determine whether we should abort the region server... -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-27352) Quoted string argument with spaces passed from command line are propagated wrongly to the underlying java class
[ https://issues.apache.org/jira/browse/HBASE-27352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-27352. -- Resolution: Fixed Committed to branch-2.5+. Thanks [~ukumar] for the contribution! > Quoted string argument with spaces passed from command line are propagated > wrongly to the underlying java class > --- > > Key: HBASE-27352 > URL: https://issues.apache.org/jira/browse/HBASE-27352 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.5.0 >Reporter: Ujjawal Kumar >Assignee: Ujjawal Kumar >Priority: Minor > Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4 > > Attachments: Screenshot 2022-08-31 at 10.52.11 PM.png > > > Running the following command : > /hbase/bin/hbase org.apache.hadoop.hbase.mapreduce.Import > -Dmapreduce.map.java.opts="-XX:+HeapDumpOnOutOfMemoryError > -XX:HeapDumpPath=$HADOOP_HOME/logs/@taskid@_map.hprof" SYSTEM.CATALOG /tmp > fails with error. It is due to the fact that [read -r -a CMD_ARGS <<< "$@" > |https://github.com/apache/hbase/blob/master/bin/hbase#L878] in the hbase > script would split the command line args (i.e. "$@"). Instead a > straightforward array copy can be used. > !Screenshot 2022-08-31 at 10.52.11 PM.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27367) Create admin/shell api for reloading just HMaster configs
Bryan Beaudreault created HBASE-27367:
-
Summary: Create admin/shell api for reloading just HMaster configs
Key: HBASE-27367
URL: https://issues.apache.org/jira/browse/HBASE-27367
Project: HBase
Issue Type: New Feature
Reporter: Bryan Beaudreault
We have {{update_config}} and {{{}update_all_config{}}}. The former can do an
individual host (RS or HMaster), the latter does all hosts in the cluster. If
you just want to reload HMaster(s) you need to go into JMX metrics to find the
tag.ServerName for each HMaster, and then paste that into individual
update_config calls.
We could either add a new {{update_hmaster_config}} or add an argument to the
existing {{{}update_all_config 'hmaster'{}}}. Whatever way we go, we should add
a corresponding method in Admin as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
