[jira] [Created] (HBASE-28227) Tables to which Stripe Compaction policy is applied cannot be forced to trigger Major Compaction.
longping_jie created HBASE-28227: Summary: Tables to which Stripe Compaction policy is applied cannot be forced to trigger Major Compaction. Key: HBASE-28227 URL: https://issues.apache.org/jira/browse/HBASE-28227 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.2.6 Reporter: longping_jie There is a table and the Stripe Compaction strategy is applied. Each region has an average value of 40G and is divided into 8 Stripes. Each Stripe is 5G. The business deletes a large amount of data. Manually triggering major compaction on the entire table and a single region does not work and cannot be selected. After reading the source code, the merging strategy applied under each Stripe is ExploringCompactionPolicy. This strategy has a key point. It filters the Store file list of a single Stripe. In the candidate file list, as long as there is a file that is too large in size and meets the condition, fileSize > (totalFileSize - fileSize) * (hbase.hstore.compaction.ratio default value 1.2), files will not be filtered out to participate in major compaction. It is necessary to support a forced merging mechanism. For scenarios where a large amount of data is deleted, or where bulkload exists, you can explicitly pass in a parameter such as foreMajor when manually triggering the major, and then perform forced Major Compaction in Stripe units to support the data. Clean up. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28226) The flush table operation encountered an NPE and failed.
Haiping lv created HBASE-28226: -- Summary: The flush table operation encountered an NPE and failed. Key: HBASE-28226 URL: https://issues.apache.org/jira/browse/HBASE-28226 Project: HBase Issue Type: Bug Affects Versions: 3.0.0-beta-1 Reporter: Haiping lv Assignee: Haiping lv Executing a merge region before the flush table is completed will result in an NPE error for the flush table and cause it to fail Triggering logic # Execute the flush table command. # Merge regions before the flush is completed. # After the merge is completed, the two parent regions will be cleaned up. # An NPE will be reported when the flush operation is performed on that parent region. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28225) Bump cryptography from 41.0.4 to 41.0.6 in /dev-support/git-jira-release-audit
[ https://issues.apache.org/jira/browse/HBASE-28225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28225. --- Fix Version/s: 4.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed Merged to master. > Bump cryptography from 41.0.4 to 41.0.6 in > /dev-support/git-jira-release-audit > --- > > Key: HBASE-28225 > URL: https://issues.apache.org/jira/browse/HBASE-28225 > Project: HBase > Issue Type: Task > Components: scripts, security >Reporter: Duo Zhang >Priority: Major > Fix For: 4.0.0-alpha-1 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28225) Bump cryptography from 41.0.4 to 41.0.6 in /dev-support/git-jira-release-audit
Duo Zhang created HBASE-28225: - Summary: Bump cryptography from 41.0.4 to 41.0.6 in /dev-support/git-jira-release-audit Key: HBASE-28225 URL: https://issues.apache.org/jira/browse/HBASE-28225 Project: HBase Issue Type: Task Components: scripts, security Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28224) ClientSideRegionScanner appears not to shutdown MobFileCache
Mark Hale created HBASE-28224: - Summary: ClientSideRegionScanner appears not to shutdown MobFileCache Key: HBASE-28224 URL: https://issues.apache.org/jira/browse/HBASE-28224 Project: HBase Issue Type: Bug Components: Scanners Affects Versions: 2.5.5 Reporter: Mark Hale As far as I can tell from the code, and also what I see in jstack, ClientSideRegionScanner doesn't appear to shutdown it's MobFileCache. After doing a lot of scanning of a snapshot, application threads are polluted with a lot of MobFileCache ScheduledThreadPoolExecutors that never seem to get cleaned up. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28217) PrefetchExecutor should not run for files from CFs that have disabled BLOCKCACHE
[ https://issues.apache.org/jira/browse/HBASE-28217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil resolved HBASE-28217. -- Resolution: Fixed Thanks for reviewing it, [~psomogyi]. I have merged it to master, branch-3, branch-2, branch-2.5 and branch-2.4. > PrefetchExecutor should not run for files from CFs that have disabled > BLOCKCACHE > > > Key: HBASE-28217 > URL: https://issues.apache.org/jira/browse/HBASE-28217 > Project: HBase > Issue Type: Bug >Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.6, 4.0.0-alpha-1 >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Major > Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 4.0.0-alpha-1, 2.5.7 > > > HFilePReadReader relies on the return of CacheConfig.shouldPrefetchOnOpen > return to decide if it should run the PrefetchExecutor for the files. > Currently, CacheConfig.shouldPrefetchOnOpen returns true if > "hbase.rs.prefetchblocksonopen" is set to true at the config, OR > PREFETCH_BLOCKS_ON_OPEN is set to true at CF level. > There's also the CacheConfig.shouldCacheDataOnRead, which returns true if > both hbase.block.data.cacheonread is set to true at the config AND BLOCKCACHE > is set to true at CF level. > If BLOCKCACHE is set to false at CF level, HFilePReadReader will still run > the PrefetchExecutor to read all the file's blocks from the FileSystem, but > then would find out the given block shouldn't be cached. > I believe we should change CacheConfig.shouldPrefetchOnOpen to return true > only if CacheConfig.shouldCacheDataOnRead is also true. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28223) Including shaded netty-all in hbase-shaded-mapreduce
Andor Molnar created HBASE-28223: Summary: Including shaded netty-all in hbase-shaded-mapreduce Key: HBASE-28223 URL: https://issues.apache.org/jira/browse/HBASE-28223 Project: HBase Issue Type: Improvement Components: mapreduce, shading Reporter: Andor Molnar Assignee: Andor Molnar Since ZooKeeper TLS connection is supported within HBase, we need netty-all on the classpath. It's already present in hbase-shaded-client and we need it in hbase-shaded-mapreduce for MR clients too. netty-all is already included in pom.xml, but only with test-scope, so shading plugin doesn't add it to the jar. Patch will override it to compile-scope. -- This message was sent by Atlassian Jira (v8.20.10#820010)
