[jira] [Resolved] (HBASE-24915) Improve BlockCache read performance by specifying BlockType
[
https://issues.apache.org/jira/browse/HBASE-24915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-24915.
---
Hadoop Flags: Reviewed
Resolution: Fixed
Added you as a contributor [~fanrui] . Thanks for the patch (Thanks for review
[~vjasani] )
> Improve BlockCache read performance by specifying BlockType
> ---
>
> Key: HBASE-24915
> URL: https://issues.apache.org/jira/browse/HBASE-24915
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, Performance
>Reporter: fanrui
>Assignee: fanrui
>Priority: Major
>
> CombinedBlockCache contains l1Cache and l2Cache. l1Cache stores MetaBlock and
> l2Cache stores DataBlock. Because getBlock does not know the BlockType, the
> getBlock of CombinedBlockCache queries l1Cache first, and then l2Cache. But
> actually querying DataBlock is not necessary to query l1Cache.
> Therefore, in some cases where BlockType is known, BlockCache read
> performance can be improved.
> h2. Code：
> BlockCache: default call old getBlock
> {code:java}
> default Cacheable getBlock(BlockCacheKey cacheKey, boolean caching, boolean
> repeat,
> boolean updateCacheMetrics, BlockType blockType) {
> return getBlock(cacheKey, caching, repeat, updateCacheMetrics);
> }
> {code}
> CombinedBlockCache:
> {code:java}
> @Override
> public Cacheable getBlock(BlockCacheKey cacheKey, boolean caching, boolean
> repeat,
> boolean updateCacheMetrics, BlockType blockType) {
> if (blockType == null) {
> return getBlock(cacheKey, caching, repeat, updateCacheMetrics);
> }
> boolean metaBlock = isMetaBlock(blockType);
> if (metaBlock) {
> return l1Cache.getBlock(cacheKey, caching, repeat, updateCacheMetrics);
> } else {
> return l2Cache.getBlock(cacheKey, caching, repeat, updateCacheMetrics);
> }
> }private boolean isMetaBlock(BlockType blockType) {
> return blockType.getCategory() != BlockCategory.DATA;
> }
> {code}
> HFileReaderImpl#getCachedBlock call BlockCache#getBlock(XXX,
> expectedBlockType)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-24918) Make RegionInfo#UNDEFINED IA.Private
[ https://issues.apache.org/jira/browse/HBASE-24918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-24918. --- Hadoop Flags: Reviewed Resolution: Fixed Merged PR. Thanks for patience and reivews [~bharathv] , [~vjasani] , and [~zhangduo] > Make RegionInfo#UNDEFINED IA.Private > > > Key: HBASE-24918 > URL: https://issues.apache.org/jira/browse/HBASE-24918 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.4.0, 2.3.2 >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.2 > > > From review of parent PR, RI#UNDEFINED is for internal use only... note it w/ > a IA.Private marking. Also deprecate it so we have option to remove as its > loading implicated in the parent issue static load deadlock. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24861) Release 2.3.1
[ https://issues.apache.org/jira/browse/HBASE-24861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-24861. -- Resolution: Done > Release 2.3.1 > - > > Key: HBASE-24861 > URL: https://issues.apache.org/jira/browse/HBASE-24861 > Project: HBase > Issue Type: Task > Components: community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24868) Send announce email
[ https://issues.apache.org/jira/browse/HBASE-24868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-24868. -- Resolution: Done > Send announce email > --- > > Key: HBASE-24868 > URL: https://issues.apache.org/jira/browse/HBASE-24868 > Project: HBase > Issue Type: Sub-task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[ANNOUNCE] Apache HBase 2.3.1 is now available for download
The HBase team is happy to announce the immediate availability of HBase 2.3.1. Apache HBase™ is an open-source, distributed, versioned, non-relational database. Apache HBase gives you low latency random access to billions of rows with millions of columns atop non-specialized hardware. To learn more about HBase, see https://hbase.apache.org/. HBase 2.3.1 is the first patch release in the HBase 2.3.x line, which aims to improve the stability and reliability of HBase. This release includes 50 bug fixes and improvements since 2.3.0. The full list of issues can be found in the included CHANGES.md and RELEASENOTES.md, or via our issue tracker: https://s.apache.org/hbase-2.3.1-jira To download please follow the links and instructions on our website: https://hbase.apache.org/downloads.html Questions, comments, and problems are always welcome at: dev@hbase.apache.org. Thanks to all who contributed and made this release possible. Cheers, The HBase Dev Team
[jira] [Resolved] (HBASE-24926) Should call setFailure in MergeTableRegionsProcedure when isMergeable returns false
[ https://issues.apache.org/jira/browse/HBASE-24926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-24926. --- Hadoop Flags: Reviewed Resolution: Fixed Pushed to branch-2.2+. Thanks [~vjasani] for reviewing. > Should call setFailure in MergeTableRegionsProcedure when isMergeable returns > false > --- > > Key: HBASE-24926 > URL: https://issues.apache.org/jira/browse/HBASE-24926 > Project: HBase > Issue Type: Bug > Components: master, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24686) [LOG] Log improvement in Connection#close
[ https://issues.apache.org/jira/browse/HBASE-24686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani resolved HBASE-24686. -- Fix Version/s: 2.3.2 2.2.7 2.4.0 3.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed > [LOG] Log improvement in Connection#close > - > > Key: HBASE-24686 > URL: https://issues.apache.org/jira/browse/HBASE-24686 > Project: HBase > Issue Type: Improvement > Components: Client, logging >Affects Versions: 2.2.3 >Reporter: mokai >Assignee: mokai >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.2 > > > We met some customers used hbase connection improperly, some threads call > failed since the shared connection closed by one of the threads. > It's better to print the details when connection closing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24883) Migrate hbase-connectors testing to ci-hadoop
[ https://issues.apache.org/jira/browse/HBASE-24883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-24883. --- Fix Version/s: connector-1.0.1 Resolution: Fixed Merged to master and changed repository URL on ci-hadoop. Thanks [~zhangduo] and [~busbey] for the reviews! > Migrate hbase-connectors testing to ci-hadoop > - > > Key: HBASE-24883 > URL: https://issues.apache.org/jira/browse/HBASE-24883 > Project: HBase > Issue Type: Task > Components: build, hbase-connectors >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > Fix For: connector-1.0.1 > > > Move hbase-connectors GitHub Pull Request build to ci-hadoop Jenkins master. > The new job should be based on a Jenkinsfile similarly to the main > repository. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24926) Should call setFailure in MergeTableRegionsProcedure when isMergeable returns false
Duo Zhang created HBASE-24926: - Summary: Should call setFailure in MergeTableRegionsProcedure when isMergeable returns false Key: HBASE-24926 URL: https://issues.apache.org/jira/browse/HBASE-24926 Project: HBase Issue Type: Bug Components: master, proc-v2 Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24912) Enlarge MemstoreFlusherChore/CompactionChecker period for unit test
[ https://issues.apache.org/jira/browse/HBASE-24912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24912. Fix Version/s: 2.3.2 2.2.6 2.4.0 3.0.0-alpha-1 Resolution: Fixed Pushed to branch-2.2+. Thanks [~stack] for reviewing. > Enlarge MemstoreFlusherChore/CompactionChecker period for unit test > --- > > Key: HBASE-24912 > URL: https://issues.apache.org/jira/browse/HBASE-24912 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 > > > Too many debug logs when run unit test now. > > 2020-08-19 01:20:59,899 DEBUG [regionserver/asf909:0.Chore.1] > hbase.ScheduledChore(192): CompactionChecker execution time: 0 ms. > 2020-08-19 01:20:59,899 DEBUG [regionserver/asf909:0.Chore.1] > hbase.ScheduledChore(192): MemstoreFlusherChore execution time: 0 ms. > 2020-08-19 01:20:59,900 DEBUG [regionserver/asf909:0.Chore.1] > hbase.ScheduledChore(192): CompactionChecker execution time: 0 ms. > 2020-08-19 01:20:59,900 DEBUG [regionserver/asf909:0.Chore.1] > hbase.ScheduledChore(192): MemstoreFlusherChore execution time: 0 ms. > 2020-08-19 01:20:59,905 DEBUG [regionserver/asf909:0.Chore.1] > hbase.ScheduledChore(192): MemstoreFlusherChore execution time: 0 ms. > 2020-08-19 01:20:59,905 DEBUG [regionserver/asf909:0.Chore.1] > hbase.ScheduledChore(192): CompactionChecker execution time: 0 ms. > 2020-08-19 01:21:00,001 DEBUG [regionserver/asf909:0.Chore.1] > hbase.ScheduledChore(192): CompactionChecker execution time: 0 ms. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24919) A tool to rewrite corrupted HFiles
[ https://issues.apache.org/jira/browse/HBASE-24919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John resolved HBASE-24919. Resolution: Duplicate Dup of HBASE-24920 > A tool to rewrite corrupted HFiles > -- > > Key: HBASE-24919 > URL: https://issues.apache.org/jira/browse/HBASE-24919 > Project: HBase > Issue Type: Brainstorming > Components: hbase-operator-tools >Reporter: Andrey Elenskiy >Priority: Major > > Typically I have been dealing with corrupted HFiles (due to loss of hdfs > blocks) by just removing them. However, It always seemed wasteful to throw > away the entire HFile (which can be hundreds of gigabytes), just because one > hdfs block is missing (128MB). > I think there's a possibility for a tool that can rewrite an HFile by > skipping corrupted blocks. > There can be multiple types of issues with hdfs blocks but any of them can be > treated as if the block doesn't exist: > 1. All the replicas can be lost > 2. The block can be corrupted due to some bug in hdfs (I've recently run into > HDFS-15186 by experimenting with EC). > At the simplest the tool can be a local mapreduce job (mapper only) with a > custom HFile reader input that can seek to next DATABLK to skip corrupted > hdfs blocks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
