[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…
Apache-HBase commented on pull request #2451: URL: https://github.com/apache/hbase/pull/2451#issuecomment-701208280 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-18070 Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 20s | HBASE-18070 passed | | +1 :green_heart: | compile | 1m 52s | HBASE-18070 passed | | +1 :green_heart: | shadedjars | 9m 20s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 52s | hbase-server in HBASE-18070 failed. | | -0 :warning: | patch | 10m 24s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 16s | the patch passed | | +1 :green_heart: | compile | 1m 34s | the patch passed | | +1 :green_heart: | javac | 1m 34s | the patch passed | | +1 :green_heart: | shadedjars | 8m 56s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 56s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 248m 17s | hbase-server in the patch passed. | | | | 287m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2451 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a60eeb66e73b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-18070 / 03e8e5b517 | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/4/testReport/ | | Max. process+thread count | 2970 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] lidingsheng opened a new pull request #2474: HBASE-25034：Table regions details on master GUI display slowly.
lidingsheng opened a new pull request #2474: URL: https://github.com/apache/hbase/pull/2474 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] nyl3532016 opened a new pull request #2473: HBASE-25115 HFilePrettyPrinter can't seek to the row which is the first row of a hfile
nyl3532016 opened a new pull request #2473: URL: https://github.com/apache/hbase/pull/2473 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] lidingsheng opened a new pull request #2475: HBASE-25034:Table regions details on master GUI display slowly.
lidingsheng opened a new pull request #2475: URL: https://github.com/apache/hbase/pull/2475 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-23887) BlockCache performance improve by reduce eviction rate
[
https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204523#comment-17204523
]
Danil Lipovoy commented on HBASE-23887:
---
Thank you for interesting)
But it looks like the feature will never merged because it has dragged on for 7
month and more then dozen developers watching on this and nothing happen.
I don't understand how open source development works, sometimes some created
issue like [HBASE-24915|https://issues.apache.org/jira/browse/HBASE-24915]
which save 1% CPU and it is apply without discussion in 3 days. But in this
case something going wrong and I have no idea why.
> BlockCache performance improve by reduce eviction rate
> --
>
> Key: HBASE-23887
> URL: https://issues.apache.org/jira/browse/HBASE-23887
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, Performance
>Reporter: Danil Lipovoy
>Assignee: Danil Lipovoy
>Priority: Minor
> Attachments: 1582787018434_rs_metrics.jpg,
> 1582801838065_rs_metrics_new.png, BC_LongRun.png,
> BlockCacheEvictionProcess.gif, BlockCacheEvictionProcess.gif, cmp.png,
> evict_BC100_vs_BC23.png, eviction_100p.png, eviction_100p.png,
> eviction_100p.png, gc_100p.png, graph.png, image-2020-06-07-08-11-11-929.png,
> image-2020-06-07-08-19-00-922.png, image-2020-06-07-12-07-24-903.png,
> image-2020-06-07-12-07-30-307.png, image-2020-06-08-17-38-45-159.png,
> image-2020-06-08-17-38-52-579.png, image-2020-06-08-18-35-48-366.png,
> image-2020-06-14-20-51-11-905.png, image-2020-06-22-05-57-45-578.png,
> image-2020-09-23-09-48-59-714.png, image-2020-09-23-10-06-11-189.png,
> ratio.png, ratio2.png, read_requests_100pBC_vs_23pBC.png, requests_100p.png,
> requests_100p.png, requests_new2_100p.png, requests_new_100p.png, scan.png,
> scan_and_gets.png, scan_and_gets2.png, wave.png, ycsb_logs.zip
>
>
> Hi!
> I first time here, correct me please if something wrong.
> All latest information is here:
> [https://docs.google.com/document/d/1X8jVnK_3lp9ibpX6lnISf_He-6xrHZL0jQQ7hoTV0-g/edit?usp=sharing]
> I want propose how to improve performance when data in HFiles much more than
> BlockChache (usual story in BigData). The idea - caching only part of DATA
> blocks. It is good becouse LruBlockCache starts to work and save huge amount
> of GC.
> Sometimes we have more data than can fit into BlockCache and it is cause a
> high rate of evictions. In this case we can skip cache a block N and insted
> cache the N+1th block. Anyway we would evict N block quite soon and that why
> that skipping good for performance.
> ---
> Some information below isn't actual
> ---
>
>
> Example:
> Imagine we have little cache, just can fit only 1 block and we are trying to
> read 3 blocks with offsets:
> 124
> 198
> 223
> Current way - we put the block 124, then put 198, evict 124, put 223, evict
> 198. A lot of work (5 actions).
> With the feature - last few digits evenly distributed from 0 to 99. When we
> divide by modulus we got:
> 124 -> 24
> 198 -> 98
> 223 -> 23
> It helps to sort them. Some part, for example below 50 (if we set
> *hbase.lru.cache.data.block.percent* = 50) go into the cache. And skip
> others. It means we will not try to handle the block 198 and save CPU for
> other job. In the result - we put block 124, then put 223, evict 124 (3
> actions).
> See the picture in attachment with test below. Requests per second is higher,
> GC is lower.
>
> The key point of the code:
> Added the parameter: *hbase.lru.cache.data.block.percent* which by default =
> 100
>
> But if we set it 1-99, then will work the next logic:
>
>
> {code:java}
> public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean
> inMemory) {
> if (cacheDataBlockPercent != 100 && buf.getBlockType().isData())
> if (cacheKey.getOffset() % 100 >= cacheDataBlockPercent)
> return;
> ...
> // the same code as usual
> }
> {code}
>
> Other parameters help to control when this logic will be enabled. It means it
> will work only while heavy reading going on.
> hbase.lru.cache.heavy.eviction.count.limit - set how many times have to run
> eviction process that start to avoid of putting data to BlockCache
> hbase.lru.cache.heavy.eviction.bytes.size.limit - set how many bytes have to
> evicted each time that start to avoid of putting data to BlockCache
> By default: if 10 times (100 secunds) evicted more than 10 MB (each time)
> then we start to skip 50% of data blocks.
> When heavy evitions process end then new logic off and will put into
> BlockCache all blocks again.
>
> Descriptions of the test:
> 4 nodes E5-2698 v4 @ 2.20GHz, 700 Gb Mem.
> 4 RegionServers
> 4 tables by 64 regions by 1.88 Gb data in each = 600 Gb total (onl
[GitHub] [hbase] lidingsheng closed pull request #2474: HBASE-25034：Table regions details on master GUI display slowly.
lidingsheng closed pull request #2474: URL: https://github.com/apache/hbase/pull/2474 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-23887) BlockCache performance improve by reduce eviction rate
[
https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204523#comment-17204523
]
Danil Lipovoy edited comment on HBASE-23887 at 9/30/20, 7:41 AM:
-
Thank you for interesting)
But it looks like the feature will never merged because it has dragged on for 7
month and more then dozen developers watching on this and nothing happen.
I don't understand how open source development works, sometimes some created
issue like HBASE-24915 which save 1% CPU and it is apply without discussion in
1 day. But in this case something going wrong and I have no idea why.
was (Author: pustota):
Thank you for interesting)
But it looks like the feature will never merged because it has dragged on for 7
month and more then dozen developers watching on this and nothing happen.
I don't understand how open source development works, sometimes some created
issue like [HBASE-24915|https://issues.apache.org/jira/browse/HBASE-24915]
which save 1% CPU and it is apply without discussion in 3 days. But in this
case something going wrong and I have no idea why.
> BlockCache performance improve by reduce eviction rate
> --
>
> Key: HBASE-23887
> URL: https://issues.apache.org/jira/browse/HBASE-23887
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, Performance
>Reporter: Danil Lipovoy
>Assignee: Danil Lipovoy
>Priority: Minor
> Attachments: 1582787018434_rs_metrics.jpg,
> 1582801838065_rs_metrics_new.png, BC_LongRun.png,
> BlockCacheEvictionProcess.gif, BlockCacheEvictionProcess.gif, cmp.png,
> evict_BC100_vs_BC23.png, eviction_100p.png, eviction_100p.png,
> eviction_100p.png, gc_100p.png, graph.png, image-2020-06-07-08-11-11-929.png,
> image-2020-06-07-08-19-00-922.png, image-2020-06-07-12-07-24-903.png,
> image-2020-06-07-12-07-30-307.png, image-2020-06-08-17-38-45-159.png,
> image-2020-06-08-17-38-52-579.png, image-2020-06-08-18-35-48-366.png,
> image-2020-06-14-20-51-11-905.png, image-2020-06-22-05-57-45-578.png,
> image-2020-09-23-09-48-59-714.png, image-2020-09-23-10-06-11-189.png,
> ratio.png, ratio2.png, read_requests_100pBC_vs_23pBC.png, requests_100p.png,
> requests_100p.png, requests_new2_100p.png, requests_new_100p.png, scan.png,
> scan_and_gets.png, scan_and_gets2.png, wave.png, ycsb_logs.zip
>
>
> Hi!
> I first time here, correct me please if something wrong.
> All latest information is here:
> [https://docs.google.com/document/d/1X8jVnK_3lp9ibpX6lnISf_He-6xrHZL0jQQ7hoTV0-g/edit?usp=sharing]
> I want propose how to improve performance when data in HFiles much more than
> BlockChache (usual story in BigData). The idea - caching only part of DATA
> blocks. It is good becouse LruBlockCache starts to work and save huge amount
> of GC.
> Sometimes we have more data than can fit into BlockCache and it is cause a
> high rate of evictions. In this case we can skip cache a block N and insted
> cache the N+1th block. Anyway we would evict N block quite soon and that why
> that skipping good for performance.
> ---
> Some information below isn't actual
> ---
>
>
> Example:
> Imagine we have little cache, just can fit only 1 block and we are trying to
> read 3 blocks with offsets:
> 124
> 198
> 223
> Current way - we put the block 124, then put 198, evict 124, put 223, evict
> 198. A lot of work (5 actions).
> With the feature - last few digits evenly distributed from 0 to 99. When we
> divide by modulus we got:
> 124 -> 24
> 198 -> 98
> 223 -> 23
> It helps to sort them. Some part, for example below 50 (if we set
> *hbase.lru.cache.data.block.percent* = 50) go into the cache. And skip
> others. It means we will not try to handle the block 198 and save CPU for
> other job. In the result - we put block 124, then put 223, evict 124 (3
> actions).
> See the picture in attachment with test below. Requests per second is higher,
> GC is lower.
>
> The key point of the code:
> Added the parameter: *hbase.lru.cache.data.block.percent* which by default =
> 100
>
> But if we set it 1-99, then will work the next logic:
>
>
> {code:java}
> public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean
> inMemory) {
> if (cacheDataBlockPercent != 100 && buf.getBlockType().isData())
> if (cacheKey.getOffset() % 100 >= cacheDataBlockPercent)
> return;
> ...
> // the same code as usual
> }
> {code}
>
> Other parameters help to control when this logic will be enabled. It means it
> will work only while heavy reading going on.
> hbase.lru.cache.heavy.eviction.count.limit - set how many times have to run
> eviction process that start to avoid of putting data to BlockCache
> hbase.lru.cache.heavy.eviction.bytes.size
[jira] [Comment Edited] (HBASE-23887) BlockCache performance improve by reduce eviction rate
[
https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204523#comment-17204523
]
Danil Lipovoy edited comment on HBASE-23887 at 9/30/20, 7:48 AM:
-
Thank you for interesting)
But it looks like the feature will never merged because it has dragged on for 7
month and more then dozen developers watching on this and nothing happen.
I don't understand how open source development works, sometimes some created
issue like HBASE-24915 which save 1% CPU and it is apply without discussion in
1 day. But in this case something going wrong and I have no idea why. Maybe
because HBASE-24915 has priority "major" but I set just "minor";))
was (Author: pustota):
Thank you for interesting)
But it looks like the feature will never merged because it has dragged on for 7
month and more then dozen developers watching on this and nothing happen.
I don't understand how open source development works, sometimes some created
issue like HBASE-24915 which save 1% CPU and it is apply without discussion in
1 day. But in this case something going wrong and I have no idea why.
> BlockCache performance improve by reduce eviction rate
> --
>
> Key: HBASE-23887
> URL: https://issues.apache.org/jira/browse/HBASE-23887
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, Performance
>Reporter: Danil Lipovoy
>Assignee: Danil Lipovoy
>Priority: Minor
> Attachments: 1582787018434_rs_metrics.jpg,
> 1582801838065_rs_metrics_new.png, BC_LongRun.png,
> BlockCacheEvictionProcess.gif, BlockCacheEvictionProcess.gif, cmp.png,
> evict_BC100_vs_BC23.png, eviction_100p.png, eviction_100p.png,
> eviction_100p.png, gc_100p.png, graph.png, image-2020-06-07-08-11-11-929.png,
> image-2020-06-07-08-19-00-922.png, image-2020-06-07-12-07-24-903.png,
> image-2020-06-07-12-07-30-307.png, image-2020-06-08-17-38-45-159.png,
> image-2020-06-08-17-38-52-579.png, image-2020-06-08-18-35-48-366.png,
> image-2020-06-14-20-51-11-905.png, image-2020-06-22-05-57-45-578.png,
> image-2020-09-23-09-48-59-714.png, image-2020-09-23-10-06-11-189.png,
> ratio.png, ratio2.png, read_requests_100pBC_vs_23pBC.png, requests_100p.png,
> requests_100p.png, requests_new2_100p.png, requests_new_100p.png, scan.png,
> scan_and_gets.png, scan_and_gets2.png, wave.png, ycsb_logs.zip
>
>
> Hi!
> I first time here, correct me please if something wrong.
> All latest information is here:
> [https://docs.google.com/document/d/1X8jVnK_3lp9ibpX6lnISf_He-6xrHZL0jQQ7hoTV0-g/edit?usp=sharing]
> I want propose how to improve performance when data in HFiles much more than
> BlockChache (usual story in BigData). The idea - caching only part of DATA
> blocks. It is good becouse LruBlockCache starts to work and save huge amount
> of GC.
> Sometimes we have more data than can fit into BlockCache and it is cause a
> high rate of evictions. In this case we can skip cache a block N and insted
> cache the N+1th block. Anyway we would evict N block quite soon and that why
> that skipping good for performance.
> ---
> Some information below isn't actual
> ---
>
>
> Example:
> Imagine we have little cache, just can fit only 1 block and we are trying to
> read 3 blocks with offsets:
> 124
> 198
> 223
> Current way - we put the block 124, then put 198, evict 124, put 223, evict
> 198. A lot of work (5 actions).
> With the feature - last few digits evenly distributed from 0 to 99. When we
> divide by modulus we got:
> 124 -> 24
> 198 -> 98
> 223 -> 23
> It helps to sort them. Some part, for example below 50 (if we set
> *hbase.lru.cache.data.block.percent* = 50) go into the cache. And skip
> others. It means we will not try to handle the block 198 and save CPU for
> other job. In the result - we put block 124, then put 223, evict 124 (3
> actions).
> See the picture in attachment with test below. Requests per second is higher,
> GC is lower.
>
> The key point of the code:
> Added the parameter: *hbase.lru.cache.data.block.percent* which by default =
> 100
>
> But if we set it 1-99, then will work the next logic:
>
>
> {code:java}
> public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean
> inMemory) {
> if (cacheDataBlockPercent != 100 && buf.getBlockType().isData())
> if (cacheKey.getOffset() % 100 >= cacheDataBlockPercent)
> return;
> ...
> // the same code as usual
> }
> {code}
>
> Other parameters help to control when this logic will be enabled. It means it
> will work only while heavy reading going on.
> hbase.lru.cache.heavy.eviction.count.limit - set how many times have to run
> eviction process that start to avoid of putting data to BlockCache
> hbase.lru.cache.heavy.
[GitHub] [hbase] Apache-HBase commented on pull request #2472: HBASE-25122 [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory
Apache-HBase commented on pull request #2472: URL: https://github.com/apache/hbase/pull/2472#issuecomment-701231339 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 6s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 10m 30s | branch-1 passed | | +1 :green_heart: | compile | 0m 53s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | compile | 0m 54s | branch-1 passed with JDK v1.7.0_272 | | +1 :green_heart: | checkstyle | 2m 11s | branch-1 passed | | +1 :green_heart: | shadedjars | 4m 25s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 52s | branch-1 passed with JDK v1.7.0_272 | | +0 :ok: | spotbugs | 3m 56s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 52s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 28s | the patch passed | | +1 :green_heart: | compile | 0m 56s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javac | 0m 56s | the patch passed | | +1 :green_heart: | compile | 0m 54s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | javac | 0m 54s | the patch passed | | +1 :green_heart: | checkstyle | 1m 56s | hbase-server: The patch generated 0 new + 3 unchanged - 3 fixed = 3 total (was 6) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 41s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 5m 40s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 43s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 52s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | findbugs | 4m 26s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 149m 43s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 42s | The patch does not generate ASF License warnings. | | | | 207m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2472/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2472 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 86ce5299aec8 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2472/out/precommit/personality/provided.sh | | git revision | branch-1 / e719a5b | | Default Java | 1.7.0_272 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2472/1/testReport/ | | Max. process+thread count | 3875 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2472/1/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2473: HBASE-25115 HFilePrettyPrinter can't seek to the row which is the first row of a hfile
Apache-HBase commented on pull request #2473: URL: https://github.com/apache/hbase/pull/2473#issuecomment-701233888 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 28s | master passed | | +1 :green_heart: | checkstyle | 1m 22s | master passed | | +1 :green_heart: | spotbugs | 2m 34s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 50s | the patch passed | | +1 :green_heart: | checkstyle | 1m 6s | hbase-server: The patch generated 0 new + 10 unchanged - 1 fixed = 10 total (was 11) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 34s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 30s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 38m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2473 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux e49d48381835 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #2472: HBASE-25122 [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory
virajjasani commented on a change in pull request #2472:
URL: https://github.com/apache/hbase/pull/2472#discussion_r497331264
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshotWithTemporaryDirectory.java
##
@@ -31,24 +27,21 @@
@Category({MediumTests.class})
public class TestExportSnapshotWithTemporaryDirectory extends
TestExportSnapshot {
- protected static String TEMP_DIR = Paths.get("").toAbsolutePath().toString()
+ Path.SEPARATOR
- + UUID.randomUUID().toString();
-
@BeforeClass
public static void setUpBeforeClass() throws Exception {
-setUpBaseConf(TEST_UTIL.getConfiguration());
+Configuration conf = TEST_UTIL.getConfiguration();
+TestExportSnapshot.setUpBaseConf(conf);
TEST_UTIL.startMiniCluster(3);
+Path rootDir =
TEST_UTIL.getMiniHBaseCluster().getMaster().getMasterFileSystem().getRootDir();
+LOG.info("Root dir: " + rootDir);
+conf.set(SnapshotDescriptionUtils.SNAPSHOT_WORKING_DIR,
+ new Path(rootDir.getParent(), ".tmpdir").toUri().toString());
Review comment:
Agree, this is reasonable.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache9 opened a new pull request #2476: HBASE-25121 Refactor MetaTableAccessor.addRegionsToMeta and its usage…
Apache9 opened a new pull request #2476: URL: https://github.com/apache/hbase/pull/2476 … places This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] lidingsheng closed pull request #2475: HBASE-25034:Table regions details on master GUI display slowly.
lidingsheng closed pull request #2475: URL: https://github.com/apache/hbase/pull/2475 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25082) Per table WAL metrics: appendCount and appendSize
[ https://issues.apache.org/jira/browse/HBASE-25082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204570#comment-17204570 ] Reid Chan commented on HBASE-25082: --- Occasionally bump in, and just an ask: for the append size why don't we use SizeHistogram but Counter. The latter will accumulate the value only > Per table WAL metrics: appendCount and appendSize > - > > Key: HBASE-25082 > URL: https://issues.apache.org/jira/browse/HBASE-25082 > Project: HBase > Issue Type: Improvement > Components: metrics >Affects Versions: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0 >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Major > Labels: observability > Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0 > > > We were trying to drill down into WAL events at a table scope and figured > that appendCount and appendSize are not exposed at a table scope. This would > be a helpful metric. > I ran the WAL benchmark tool with 10 threads and didn't see any measurable > performance difference with the attached patch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25099) Change meta replica number by altering meta table descriptor
[ https://issues.apache.org/jira/browse/HBASE-25099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204579#comment-17204579 ] Duo Zhang commented on HBASE-25099: --- Oh, there is a problem, that we can only changing the region replica count by disabling the table, but meta can not be disabled. There are two possible solutions: 1. add the support to disable meta. 2. support changing region replica number without disabling table first. WDYT [~stack]? For me, I think solution #2 is better to users, but the implementation will be more complicated. Thanks. > Change meta replica number by altering meta table descriptor > > > Key: HBASE-25099 > URL: https://issues.apache.org/jira/browse/HBASE-25099 > Project: HBase > Issue Type: Improvement > Components: meta, read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > As now we support altering meta table, it will be better to also deal with > changing meta replica number using altering meta, i.e, we could unify the > logic in MasterMetaBootstrap to ModifyTableProcedure, and another benefit is > that we do not need to restart master when changing the replica number for > meta. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] lidingsheng opened a new pull request #2477: HBASE-25034:Table regions details on master GUI display slowly.
lidingsheng opened a new pull request #2477: URL: https://github.com/apache/hbase/pull/2477 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani opened a new pull request #2478: HBASE-24492 : Remove infinite loop from ProtobufLogReader#readNext (#…
virajjasani opened a new pull request #2478: URL: https://github.com/apache/hbase/pull/2478 …1831) Signed-off-by: Duo Zhang This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani opened a new pull request #2479: HBASE-24492 : Remove infinite loop from ProtobufLogReader#readNext (#…
virajjasani opened a new pull request #2479: URL: https://github.com/apache/hbase/pull/2479 …1831) Signed-off-by: Duo Zhang This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-701261021 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 51s | master passed | | +1 :green_heart: | compile | 1m 24s | master passed | | +1 :green_heart: | shadedjars | 6m 54s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 42s | the patch passed | | +1 :green_heart: | compile | 1m 23s | the patch passed | | +1 :green_heart: | javac | 1m 23s | the patch passed | | +1 :green_heart: | shadedjars | 6m 57s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 55s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 55s | hbase-procedure in the patch passed. | | +1 :green_heart: | unit | 148m 0s | hbase-server in the patch passed. | | | | 179m 19s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/6/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6dd58a39adc5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/6/testReport/ | | Max. process+thread count | 3913 (vs. ulimit of 3) | | modules | C: hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/6/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] lidingsheng closed pull request #2477: HBASE-25034:Table regions details on master GUI display slowly.
lidingsheng closed pull request #2477: URL: https://github.com/apache/hbase/pull/2477 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2476: HBASE-25121 Refactor MetaTableAccessor.addRegionsToMeta and its usage…
Apache-HBase commented on pull request #2476: URL: https://github.com/apache/hbase/pull/2476#issuecomment-701264449 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 42s | master passed | | +1 :green_heart: | checkstyle | 1m 50s | master passed | | +1 :green_heart: | spotbugs | 3m 25s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 24s | the patch passed | | -0 :warning: | checkstyle | 1m 8s | hbase-server: The patch generated 1 new + 158 unchanged - 9 fixed = 159 total (was 167) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 19s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 55s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 39s | The patch does not generate ASF License warnings. | | | | 38m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2476 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux e7584d5b8ec0 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-client hbase-balancer hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25115) HFilePrettyPrinter can't seek to the row which is the first row of a hfile
[
https://issues.apache.org/jira/browse/HBASE-25115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chaoqiang Zhong updated HBASE-25115:
Attachment: HBASE-25115.patch
> HFilePrettyPrinter can't seek to the row which is the first row of a hfile
> --
>
> Key: HBASE-25115
> URL: https://issues.apache.org/jira/browse/HBASE-25115
> Project: HBase
> Issue Type: Bug
> Components: HFile, tooling
>Affects Versions: 2.2.6
>Reporter: Chaoqiang Zhong
>Assignee: niuyulin
>Priority: Major
> Attachments: HBASE-25115.patch
>
>
> This issue can be reproduced by below steps:
> * make a hfile contains two rows '000' and '001';
> {code:java}
> D:\bin>hbase hfile -p -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> K: 000/f1:a/1601282789548/Put/vlen=4/seqid=4 V:
> K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5 V:
> Scanned kv count -> 2{code}
> * '001' can be seeked to;
> {code:java}
> D:\bin>hbase hfile -e -w 001 -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5
> Scanned kv count -> 1{code}
> * but '000' can't be seeked to;
> {code:java}
> D:\bin>hbase hfile -e -w 000 -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> Scanned kv count -> 0{code}
> In HFilePrettyPrinter we use
> "scanner.seekTo(PrivateCellUtil.createFirstOnRow(this.row))" to seek to
> row.But this method will retrurn -1 when the row is the first row of hfile.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25115) HFilePrettyPrinter can't seek to the row which is the first row of a hfile
[
https://issues.apache.org/jira/browse/HBASE-25115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204592#comment-17204592
]
Chaoqiang Zhong commented on HBASE-25115:
-
a simple fix for this issue.
> HFilePrettyPrinter can't seek to the row which is the first row of a hfile
> --
>
> Key: HBASE-25115
> URL: https://issues.apache.org/jira/browse/HBASE-25115
> Project: HBase
> Issue Type: Bug
> Components: HFile, tooling
>Affects Versions: 2.2.6
>Reporter: Chaoqiang Zhong
>Assignee: niuyulin
>Priority: Major
> Attachments: HBASE-25115.patch
>
>
> This issue can be reproduced by below steps:
> * make a hfile contains two rows '000' and '001';
> {code:java}
> D:\bin>hbase hfile -p -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> K: 000/f1:a/1601282789548/Put/vlen=4/seqid=4 V:
> K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5 V:
> Scanned kv count -> 2{code}
> * '001' can be seeked to;
> {code:java}
> D:\bin>hbase hfile -e -w 001 -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5
> Scanned kv count -> 1{code}
> * but '000' can't be seeked to;
> {code:java}
> D:\bin>hbase hfile -e -w 000 -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> Scanned kv count -> 0{code}
> In HFilePrettyPrinter we use
> "scanner.seekTo(PrivateCellUtil.createFirstOnRow(this.row))" to seek to
> row.But this method will retrurn -1 when the row is the first row of hfile.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] lidingsheng opened a new pull request #2480: HBASE-25034:Table regions details on master GUI display slowly.
lidingsheng opened a new pull request #2480: URL: https://github.com/apache/hbase/pull/2480 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ZhongChaoqiang opened a new pull request #2481: HBASE-25115 HFilePrettyPrinter can't seek to the row which is the first row of a hfile
ZhongChaoqiang opened a new pull request #2481: URL: https://github.com/apache/hbase/pull/2481 …st row of a hfile This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] lidingsheng closed pull request #2480: HBASE-25034:Table regions details on master GUI display slowly.
lidingsheng closed pull request #2480: URL: https://github.com/apache/hbase/pull/2480 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] lidingsheng opened a new pull request #2482: HBASE-25034:Table regions details on master GUI display slowly.
lidingsheng opened a new pull request #2482: URL: https://github.com/apache/hbase/pull/2482 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2478: HBASE-24503 : Backport HBASE-24492 Remove infinite loop from ProtobufLogReader#readNext (#1831)
Apache-HBase commented on pull request #2478: URL: https://github.com/apache/hbase/pull/2478#issuecomment-701281640 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 13s | branch-2.3 passed | | +1 :green_heart: | checkstyle | 1m 22s | branch-2.3 passed | | +1 :green_heart: | spotbugs | 2m 30s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 38s | the patch passed | | +1 :green_heart: | checkstyle | 1m 10s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 52s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 44m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2478/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2478 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 4d10a1bea324 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / c1649a9d51 | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2478/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] wchevreuil commented on a change in pull request #2473: HBASE-25115 HFilePrettyPrinter can't seek to the row which is the first row of a hfile
wchevreuil commented on a change in pull request #2473:
URL: https://github.com/apache/hbase/pull/2473#discussion_r497387643
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java
##
@@ -323,15 +323,20 @@ public int processFile(Path file, boolean checkRootDir)
throws IOException {
HFileScanner scanner = reader.getScanner(false, false, false);
fileStats = new KeyValueStatsCollector();
boolean shouldScanKeysValues = false;
+ boolean needSeekToBegin = true;
if (this.isSeekToRow) {
// seek to the first kv on this row
-shouldScanKeysValues =
- (scanner.seekTo(PrivateCellUtil.createFirstOnRow(this.row)) != -1);
- } else {
+if (scanner.seekTo(PrivateCellUtil.createFirstOnRow(this.row)) != -1) {
+ needSeekToBegin = false;
+ shouldScanKeysValues = true;
+}
+ }
+ if (needSeekToBegin) {
Review comment:
Isn't this `scanner.seekTo(PrivateCellUtil.createFirstOnRow(this.row))`
always going to return -1, since it's fabricating a KV with no CF? If so, I
guess your logic could be simplified.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] bsglz commented on pull request #2482: HBASE-25034:Table regions details on master GUI display slowly.
bsglz commented on pull request #2482: URL: https://github.com/apache/hbase/pull/2482#issuecomment-701290185 You do not need create a new PR when you want to add more commit, just push it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2473: HBASE-25115 HFilePrettyPrinter can't seek to the row which is the first row of a hfile
Apache-HBase commented on pull request #2473: URL: https://github.com/apache/hbase/pull/2473#issuecomment-701297762 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 20s | master passed | | +1 :green_heart: | compile | 1m 10s | master passed | | +1 :green_heart: | shadedjars | 6m 46s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 42s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 4s | the patch passed | | +1 :green_heart: | compile | 1m 5s | the patch passed | | +1 :green_heart: | javac | 1m 5s | the patch passed | | +1 :green_heart: | shadedjars | 6m 38s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 40s | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 135m 40s | hbase-server in the patch failed. | | | | 164m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2473 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 2e0988be6aaf 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/1/testReport/ | | Max. process+thread count | 3489 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #2259: HBASE-24839 Backport to branch-1 HBASE-20561 The way we stop a ReplicationSource may cause the RS down
Apache9 commented on pull request #2259: URL: https://github.com/apache/hbase/pull/2259#issuecomment-701301609 The replication related code are completely different between 2.x and 1.x. For 2.x, we will abort the RS when we fail to modify zk, so the trick is how do we deal with zk failure when modifying zk. Could you please explain a bit how do we deal with zk failure on branch-1? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2473: HBASE-25115 HFilePrettyPrinter can't seek to the row which is the first row of a hfile
Apache-HBase commented on pull request #2473: URL: https://github.com/apache/hbase/pull/2473#issuecomment-701302769 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 17s | master passed | | +1 :green_heart: | compile | 1m 3s | master passed | | +1 :green_heart: | shadedjars | 7m 24s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 0s | the patch passed | | +1 :green_heart: | compile | 1m 9s | the patch passed | | +1 :green_heart: | javac | 1m 9s | the patch passed | | +1 :green_heart: | shadedjars | 7m 19s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 145m 36s | hbase-server in the patch passed. | | | | 174m 41s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2473 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3da23651aec7 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/1/testReport/ | | Max. process+thread count | 3773 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-23887) BlockCache performance improve by reduce eviction rate
[
https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204523#comment-17204523
]
Danil Lipovoy edited comment on HBASE-23887 at 9/30/20, 10:31 AM:
--
Thank you for your interest)
But it looks like the feature will never merged because it has dragged on for 7
month and more then dozen developers watching on this and nothing happen. We
are just discussing what would be good me to do (more performance tests,
resolve conflicts etc) and I can't see the end of that.
I don't understand how open source development works, sometimes some created
issue like HBASE-24915 which save 1% CPU and it is apply without discussion in
1 day. But in this case although is saving huge amount of CPU but something
went wrong and I have no idea why. Maybe because HBASE-24915 has priority
"major" but I set just "minor";))
was (Author: pustota):
Thank you for your interest)
But it looks like the feature will never merged because it has dragged on for 7
month and more then dozen developers watching on this and nothing happen. We
just discussing what would be good me to do (more performance tests, resolve
conflicts etc) and I can't see the end of that.
I don't understand how open source development works, sometimes some created
issue like HBASE-24915 which save 1% CPU and it is apply without discussion in
1 day. But in this case saving huge amount of CPU but something went wrong and
I have no idea why. Maybe because HBASE-24915 has priority "major" but I set
just "minor";))
> BlockCache performance improve by reduce eviction rate
> --
>
> Key: HBASE-23887
> URL: https://issues.apache.org/jira/browse/HBASE-23887
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, Performance
>Reporter: Danil Lipovoy
>Assignee: Danil Lipovoy
>Priority: Minor
> Attachments: 1582787018434_rs_metrics.jpg,
> 1582801838065_rs_metrics_new.png, BC_LongRun.png,
> BlockCacheEvictionProcess.gif, BlockCacheEvictionProcess.gif, cmp.png,
> evict_BC100_vs_BC23.png, eviction_100p.png, eviction_100p.png,
> eviction_100p.png, gc_100p.png, graph.png, image-2020-06-07-08-11-11-929.png,
> image-2020-06-07-08-19-00-922.png, image-2020-06-07-12-07-24-903.png,
> image-2020-06-07-12-07-30-307.png, image-2020-06-08-17-38-45-159.png,
> image-2020-06-08-17-38-52-579.png, image-2020-06-08-18-35-48-366.png,
> image-2020-06-14-20-51-11-905.png, image-2020-06-22-05-57-45-578.png,
> image-2020-09-23-09-48-59-714.png, image-2020-09-23-10-06-11-189.png,
> ratio.png, ratio2.png, read_requests_100pBC_vs_23pBC.png, requests_100p.png,
> requests_100p.png, requests_new2_100p.png, requests_new_100p.png, scan.png,
> scan_and_gets.png, scan_and_gets2.png, wave.png, ycsb_logs.zip
>
>
> Hi!
> I first time here, correct me please if something wrong.
> All latest information is here:
> [https://docs.google.com/document/d/1X8jVnK_3lp9ibpX6lnISf_He-6xrHZL0jQQ7hoTV0-g/edit?usp=sharing]
> I want propose how to improve performance when data in HFiles much more than
> BlockChache (usual story in BigData). The idea - caching only part of DATA
> blocks. It is good becouse LruBlockCache starts to work and save huge amount
> of GC.
> Sometimes we have more data than can fit into BlockCache and it is cause a
> high rate of evictions. In this case we can skip cache a block N and insted
> cache the N+1th block. Anyway we would evict N block quite soon and that why
> that skipping good for performance.
> ---
> Some information below isn't actual
> ---
>
>
> Example:
> Imagine we have little cache, just can fit only 1 block and we are trying to
> read 3 blocks with offsets:
> 124
> 198
> 223
> Current way - we put the block 124, then put 198, evict 124, put 223, evict
> 198. A lot of work (5 actions).
> With the feature - last few digits evenly distributed from 0 to 99. When we
> divide by modulus we got:
> 124 -> 24
> 198 -> 98
> 223 -> 23
> It helps to sort them. Some part, for example below 50 (if we set
> *hbase.lru.cache.data.block.percent* = 50) go into the cache. And skip
> others. It means we will not try to handle the block 198 and save CPU for
> other job. In the result - we put block 124, then put 223, evict 124 (3
> actions).
> See the picture in attachment with test below. Requests per second is higher,
> GC is lower.
>
> The key point of the code:
> Added the parameter: *hbase.lru.cache.data.block.percent* which by default =
> 100
>
> But if we set it 1-99, then will work the next logic:
>
>
> {code:java}
> public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean
> inMemory) {
> if (cacheDataBlockPercent != 100 && buf.getBlockType().isData())
> if (cacheKey.ge
[jira] [Comment Edited] (HBASE-23887) BlockCache performance improve by reduce eviction rate
[
https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204523#comment-17204523
]
Danil Lipovoy edited comment on HBASE-23887 at 9/30/20, 10:29 AM:
--
Thank you for your interest)
But it looks like the feature will never merged because it has dragged on for 7
month and more then dozen developers watching on this and nothing happen. We
just discussing what would be good me to do (more performance tests, resolve
conflicts etc) and I can't see the end of that.
I don't understand how open source development works, sometimes some created
issue like HBASE-24915 which save 1% CPU and it is apply without discussion in
1 day. But in this case saving huge amount of CPU but something went wrong and
I have no idea why. Maybe because HBASE-24915 has priority "major" but I set
just "minor";))
was (Author: pustota):
Thank you for interesting)
But it looks like the feature will never merged because it has dragged on for 7
month and more then dozen developers watching on this and nothing happen.
I don't understand how open source development works, sometimes some created
issue like HBASE-24915 which save 1% CPU and it is apply without discussion in
1 day. But in this case something going wrong and I have no idea why. Maybe
because HBASE-24915 has priority "major" but I set just "minor";))
> BlockCache performance improve by reduce eviction rate
> --
>
> Key: HBASE-23887
> URL: https://issues.apache.org/jira/browse/HBASE-23887
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, Performance
>Reporter: Danil Lipovoy
>Assignee: Danil Lipovoy
>Priority: Minor
> Attachments: 1582787018434_rs_metrics.jpg,
> 1582801838065_rs_metrics_new.png, BC_LongRun.png,
> BlockCacheEvictionProcess.gif, BlockCacheEvictionProcess.gif, cmp.png,
> evict_BC100_vs_BC23.png, eviction_100p.png, eviction_100p.png,
> eviction_100p.png, gc_100p.png, graph.png, image-2020-06-07-08-11-11-929.png,
> image-2020-06-07-08-19-00-922.png, image-2020-06-07-12-07-24-903.png,
> image-2020-06-07-12-07-30-307.png, image-2020-06-08-17-38-45-159.png,
> image-2020-06-08-17-38-52-579.png, image-2020-06-08-18-35-48-366.png,
> image-2020-06-14-20-51-11-905.png, image-2020-06-22-05-57-45-578.png,
> image-2020-09-23-09-48-59-714.png, image-2020-09-23-10-06-11-189.png,
> ratio.png, ratio2.png, read_requests_100pBC_vs_23pBC.png, requests_100p.png,
> requests_100p.png, requests_new2_100p.png, requests_new_100p.png, scan.png,
> scan_and_gets.png, scan_and_gets2.png, wave.png, ycsb_logs.zip
>
>
> Hi!
> I first time here, correct me please if something wrong.
> All latest information is here:
> [https://docs.google.com/document/d/1X8jVnK_3lp9ibpX6lnISf_He-6xrHZL0jQQ7hoTV0-g/edit?usp=sharing]
> I want propose how to improve performance when data in HFiles much more than
> BlockChache (usual story in BigData). The idea - caching only part of DATA
> blocks. It is good becouse LruBlockCache starts to work and save huge amount
> of GC.
> Sometimes we have more data than can fit into BlockCache and it is cause a
> high rate of evictions. In this case we can skip cache a block N and insted
> cache the N+1th block. Anyway we would evict N block quite soon and that why
> that skipping good for performance.
> ---
> Some information below isn't actual
> ---
>
>
> Example:
> Imagine we have little cache, just can fit only 1 block and we are trying to
> read 3 blocks with offsets:
> 124
> 198
> 223
> Current way - we put the block 124, then put 198, evict 124, put 223, evict
> 198. A lot of work (5 actions).
> With the feature - last few digits evenly distributed from 0 to 99. When we
> divide by modulus we got:
> 124 -> 24
> 198 -> 98
> 223 -> 23
> It helps to sort them. Some part, for example below 50 (if we set
> *hbase.lru.cache.data.block.percent* = 50) go into the cache. And skip
> others. It means we will not try to handle the block 198 and save CPU for
> other job. In the result - we put block 124, then put 223, evict 124 (3
> actions).
> See the picture in attachment with test below. Requests per second is higher,
> GC is lower.
>
> The key point of the code:
> Added the parameter: *hbase.lru.cache.data.block.percent* which by default =
> 100
>
> But if we set it 1-99, then will work the next logic:
>
>
> {code:java}
> public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean
> inMemory) {
> if (cacheDataBlockPercent != 100 && buf.getBlockType().isData())
> if (cacheKey.getOffset() % 100 >= cacheDataBlockPercent)
> return;
> ...
> // the same code as usual
> }
> {code}
>
> Other parameters help to control when this logic will be enabl
[jira] [Created] (HBASE-25123) Add possibility to set different types of L1 cache
Danil Lipovoy created HBASE-25123: - Summary: Add possibility to set different types of L1 cache Key: HBASE-25123 URL: https://issues.apache.org/jira/browse/HBASE-25123 Project: HBase Issue Type: New Feature Reporter: Danil Lipovoy The feature HBASE-23887 allow speed up to 3 times read performance but maybe it is too complicated. So there are proposals give users possibility to choose type of cache L1. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-23887) BlockCache performance improve by reduce eviction rate
[
https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204523#comment-17204523
]
Danil Lipovoy edited comment on HBASE-23887 at 9/30/20, 10:43 AM:
--
Thank you for your interest)
But it looks like the feature will never merged because it has dragged on for 7
month and more then dozen developers watching on this and nothing happen. We
are just discussing what would be good to do (more performance tests, resolve
conflicts etc) and I can't see the end of that.
I don't understand how open source development works, sometimes some created
issue like HBASE-24915 which save 1% CPU and it is apply without discussion in
1 day. But in this case although is saving huge amount of CPU but something
went wrong and I have no idea why. Maybe because HBASE-24915 has priority
"major" but I set just "minor";))
So I created HBASE-25123 and maybe somebody released possibility to set
classes L1 realisation.
was (Author: pustota):
Thank you for your interest)
But it looks like the feature will never merged because it has dragged on for 7
month and more then dozen developers watching on this and nothing happen. We
are just discussing what would be good me to do (more performance tests,
resolve conflicts etc) and I can't see the end of that.
I don't understand how open source development works, sometimes some created
issue like HBASE-24915 which save 1% CPU and it is apply without discussion in
1 day. But in this case although is saving huge amount of CPU but something
went wrong and I have no idea why. Maybe because HBASE-24915 has priority
"major" but I set just "minor";))
> BlockCache performance improve by reduce eviction rate
> --
>
> Key: HBASE-23887
> URL: https://issues.apache.org/jira/browse/HBASE-23887
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, Performance
>Reporter: Danil Lipovoy
>Assignee: Danil Lipovoy
>Priority: Minor
> Attachments: 1582787018434_rs_metrics.jpg,
> 1582801838065_rs_metrics_new.png, BC_LongRun.png,
> BlockCacheEvictionProcess.gif, BlockCacheEvictionProcess.gif, cmp.png,
> evict_BC100_vs_BC23.png, eviction_100p.png, eviction_100p.png,
> eviction_100p.png, gc_100p.png, graph.png, image-2020-06-07-08-11-11-929.png,
> image-2020-06-07-08-19-00-922.png, image-2020-06-07-12-07-24-903.png,
> image-2020-06-07-12-07-30-307.png, image-2020-06-08-17-38-45-159.png,
> image-2020-06-08-17-38-52-579.png, image-2020-06-08-18-35-48-366.png,
> image-2020-06-14-20-51-11-905.png, image-2020-06-22-05-57-45-578.png,
> image-2020-09-23-09-48-59-714.png, image-2020-09-23-10-06-11-189.png,
> ratio.png, ratio2.png, read_requests_100pBC_vs_23pBC.png, requests_100p.png,
> requests_100p.png, requests_new2_100p.png, requests_new_100p.png, scan.png,
> scan_and_gets.png, scan_and_gets2.png, wave.png, ycsb_logs.zip
>
>
> Hi!
> I first time here, correct me please if something wrong.
> All latest information is here:
> [https://docs.google.com/document/d/1X8jVnK_3lp9ibpX6lnISf_He-6xrHZL0jQQ7hoTV0-g/edit?usp=sharing]
> I want propose how to improve performance when data in HFiles much more than
> BlockChache (usual story in BigData). The idea - caching only part of DATA
> blocks. It is good becouse LruBlockCache starts to work and save huge amount
> of GC.
> Sometimes we have more data than can fit into BlockCache and it is cause a
> high rate of evictions. In this case we can skip cache a block N and insted
> cache the N+1th block. Anyway we would evict N block quite soon and that why
> that skipping good for performance.
> ---
> Some information below isn't actual
> ---
>
>
> Example:
> Imagine we have little cache, just can fit only 1 block and we are trying to
> read 3 blocks with offsets:
> 124
> 198
> 223
> Current way - we put the block 124, then put 198, evict 124, put 223, evict
> 198. A lot of work (5 actions).
> With the feature - last few digits evenly distributed from 0 to 99. When we
> divide by modulus we got:
> 124 -> 24
> 198 -> 98
> 223 -> 23
> It helps to sort them. Some part, for example below 50 (if we set
> *hbase.lru.cache.data.block.percent* = 50) go into the cache. And skip
> others. It means we will not try to handle the block 198 and save CPU for
> other job. In the result - we put block 124, then put 223, evict 124 (3
> actions).
> See the picture in attachment with test below. Requests per second is higher,
> GC is lower.
>
> The key point of the code:
> Added the parameter: *hbase.lru.cache.data.block.percent* which by default =
> 100
>
> But if we set it 1-99, then will work the next logic:
>
>
> {code:java}
> public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean
>
[jira] [Comment Edited] (HBASE-23887) BlockCache performance improve by reduce eviction rate
[ https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204523#comment-17204523 ] Danil Lipovoy edited comment on HBASE-23887 at 9/30/20, 10:44 AM: -- Thank you for your interest) But it looks like the feature will never merged because it has dragged on for 7 month and more then dozen developers watching on this and nothing happen. We are just discussing what would be good to do (more performance tests, resolve conflicts etc) and I can't see the end of that. I don't understand how open source development works, sometimes some created issue like HBASE-24915 which save 1% CPU and it is apply without discussion in 1 day. But in this case although is saving huge amount of CPU but something went wrong and I have no idea why. Maybe because HBASE-24915 has priority "major" but I set just "minor";)) So I created HBASE-25123 and maybe somebody will release possibility to set classes L1 realisation. was (Author: pustota): Thank you for your interest) But it looks like the feature will never merged because it has dragged on for 7 month and more then dozen developers watching on this and nothing happen. We are just discussing what would be good to do (more performance tests, resolve conflicts etc) and I can't see the end of that. I don't understand how open source development works, sometimes some created issue like HBASE-24915 which save 1% CPU and it is apply without discussion in 1 day. But in this case although is saving huge amount of CPU but something went wrong and I have no idea why. Maybe because HBASE-24915 has priority "major" but I set just "minor";)) So I created HBASE-25123 and maybe somebody released possibility to set classes L1 realisation. > BlockCache performance improve by reduce eviction rate > -- > > Key: HBASE-23887 > URL: https://issues.apache.org/jira/browse/HBASE-23887 > Project: HBase > Issue Type: Improvement > Components: BlockCache, Performance >Reporter: Danil Lipovoy >Assignee: Danil Lipovoy >Priority: Minor > Attachments: 1582787018434_rs_metrics.jpg, > 1582801838065_rs_metrics_new.png, BC_LongRun.png, > BlockCacheEvictionProcess.gif, BlockCacheEvictionProcess.gif, cmp.png, > evict_BC100_vs_BC23.png, eviction_100p.png, eviction_100p.png, > eviction_100p.png, gc_100p.png, graph.png, image-2020-06-07-08-11-11-929.png, > image-2020-06-07-08-19-00-922.png, image-2020-06-07-12-07-24-903.png, > image-2020-06-07-12-07-30-307.png, image-2020-06-08-17-38-45-159.png, > image-2020-06-08-17-38-52-579.png, image-2020-06-08-18-35-48-366.png, > image-2020-06-14-20-51-11-905.png, image-2020-06-22-05-57-45-578.png, > image-2020-09-23-09-48-59-714.png, image-2020-09-23-10-06-11-189.png, > ratio.png, ratio2.png, read_requests_100pBC_vs_23pBC.png, requests_100p.png, > requests_100p.png, requests_new2_100p.png, requests_new_100p.png, scan.png, > scan_and_gets.png, scan_and_gets2.png, wave.png, ycsb_logs.zip > > > Hi! > I first time here, correct me please if something wrong. > All latest information is here: > [https://docs.google.com/document/d/1X8jVnK_3lp9ibpX6lnISf_He-6xrHZL0jQQ7hoTV0-g/edit?usp=sharing] > I want propose how to improve performance when data in HFiles much more than > BlockChache (usual story in BigData). The idea - caching only part of DATA > blocks. It is good becouse LruBlockCache starts to work and save huge amount > of GC. > Sometimes we have more data than can fit into BlockCache and it is cause a > high rate of evictions. In this case we can skip cache a block N and insted > cache the N+1th block. Anyway we would evict N block quite soon and that why > that skipping good for performance. > --- > Some information below isn't actual > --- > > > Example: > Imagine we have little cache, just can fit only 1 block and we are trying to > read 3 blocks with offsets: > 124 > 198 > 223 > Current way - we put the block 124, then put 198, evict 124, put 223, evict > 198. A lot of work (5 actions). > With the feature - last few digits evenly distributed from 0 to 99. When we > divide by modulus we got: > 124 -> 24 > 198 -> 98 > 223 -> 23 > It helps to sort them. Some part, for example below 50 (if we set > *hbase.lru.cache.data.block.percent* = 50) go into the cache. And skip > others. It means we will not try to handle the block 198 and save CPU for > other job. In the result - we put block 124, then put 223, evict 124 (3 > actions). > See the picture in attachment with test below. Requests per second is higher, > GC is lower. > > The key point of the code: > Added the parameter: *hbase.lru.cache.data.block.percent* which by default = > 100 > > But if we set it 1-99, then will work the next logic: >
[jira] [Updated] (HBASE-25123) Add possibility to set different types of L1 cache
[ https://issues.apache.org/jira/browse/HBASE-25123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danil Lipovoy updated HBASE-25123: -- Description: The feature HBASE-23887 allow speed up to 3 times read performance but maybe it is too complicated. So there are proposals give users possibility to choose type of cache L1. Looks like it needs to change few classes (CombinedBlockCache, InclusiveCombinedBlockCache, CacheConfig) (was: The feature HBASE-23887 allow speed up to 3 times read performance but maybe it is too complicated. So there are proposals give users possibility to choose type of cache L1. ) > Add possibility to set different types of L1 cache > -- > > Key: HBASE-25123 > URL: https://issues.apache.org/jira/browse/HBASE-25123 > Project: HBase > Issue Type: New Feature >Reporter: Danil Lipovoy >Priority: Minor > > The feature HBASE-23887 allow speed up to 3 times read performance but maybe > it is too complicated. So there are proposals give users possibility to > choose type of cache L1. Looks like it needs to change few classes > (CombinedBlockCache, InclusiveCombinedBlockCache, CacheConfig) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25123) Add possibility to set different types of L1 cache
[ https://issues.apache.org/jira/browse/HBASE-25123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danil Lipovoy updated HBASE-25123: -- Description: The feature HBASE-23887 allow speed up to 3 times read performance but maybe it is too complicated. So there are proposals give users possibility to choose type of cache L1. Looks like it needs to change few classes (CombinedBlockCache, InclusiveCombinedBlockCache, CacheConfig) if somebody can code this it would be cool. (was: The feature HBASE-23887 allow speed up to 3 times read performance but maybe it is too complicated. So there are proposals give users possibility to choose type of cache L1. Looks like it needs to change few classes (CombinedBlockCache, InclusiveCombinedBlockCache, CacheConfig) ) > Add possibility to set different types of L1 cache > -- > > Key: HBASE-25123 > URL: https://issues.apache.org/jira/browse/HBASE-25123 > Project: HBase > Issue Type: New Feature >Reporter: Danil Lipovoy >Priority: Minor > > The feature HBASE-23887 allow speed up to 3 times read performance but maybe > it is too complicated. So there are proposals give users possibility to > choose type of cache L1. Looks like it needs to change few classes > (CombinedBlockCache, InclusiveCombinedBlockCache, CacheConfig) if somebody > can code this it would be cool. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-23887) BlockCache performance improve by reduce eviction rate
[
https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204523#comment-17204523
]
Danil Lipovoy edited comment on HBASE-23887 at 9/30/20, 10:49 AM:
--
Thank you for your interest)
But it looks like the feature will never merged because it has dragged on for 7
month and nothing happen. We are just discussing what would be good to do
(more performance tests, resolve conflicts etc) and I can't see the end of that.
I don't understand how open source development works, sometimes some created
issue like HBASE-24915 which save 1% CPU and it is apply without discussion in
1 day. But in this case although is saving huge amount of CPU but something
went wrong and I have no idea why. Maybe because HBASE-24915 has priority
"major" but I set just "minor";))
So I created HBASE-25123 and maybe somebody will release possibility to set
classes L1 realisation.
was (Author: pustota):
Thank you for your interest)
But it looks like the feature will never merged because it has dragged on for 7
month and more then dozen developers watching on this and nothing happen. We
are just discussing what would be good to do (more performance tests, resolve
conflicts etc) and I can't see the end of that.
I don't understand how open source development works, sometimes some created
issue like HBASE-24915 which save 1% CPU and it is apply without discussion in
1 day. But in this case although is saving huge amount of CPU but something
went wrong and I have no idea why. Maybe because HBASE-24915 has priority
"major" but I set just "minor";))
So I created HBASE-25123 and maybe somebody will release possibility to set
classes L1 realisation.
> BlockCache performance improve by reduce eviction rate
> --
>
> Key: HBASE-23887
> URL: https://issues.apache.org/jira/browse/HBASE-23887
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, Performance
>Reporter: Danil Lipovoy
>Assignee: Danil Lipovoy
>Priority: Minor
> Attachments: 1582787018434_rs_metrics.jpg,
> 1582801838065_rs_metrics_new.png, BC_LongRun.png,
> BlockCacheEvictionProcess.gif, BlockCacheEvictionProcess.gif, cmp.png,
> evict_BC100_vs_BC23.png, eviction_100p.png, eviction_100p.png,
> eviction_100p.png, gc_100p.png, graph.png, image-2020-06-07-08-11-11-929.png,
> image-2020-06-07-08-19-00-922.png, image-2020-06-07-12-07-24-903.png,
> image-2020-06-07-12-07-30-307.png, image-2020-06-08-17-38-45-159.png,
> image-2020-06-08-17-38-52-579.png, image-2020-06-08-18-35-48-366.png,
> image-2020-06-14-20-51-11-905.png, image-2020-06-22-05-57-45-578.png,
> image-2020-09-23-09-48-59-714.png, image-2020-09-23-10-06-11-189.png,
> ratio.png, ratio2.png, read_requests_100pBC_vs_23pBC.png, requests_100p.png,
> requests_100p.png, requests_new2_100p.png, requests_new_100p.png, scan.png,
> scan_and_gets.png, scan_and_gets2.png, wave.png, ycsb_logs.zip
>
>
> Hi!
> I first time here, correct me please if something wrong.
> All latest information is here:
> [https://docs.google.com/document/d/1X8jVnK_3lp9ibpX6lnISf_He-6xrHZL0jQQ7hoTV0-g/edit?usp=sharing]
> I want propose how to improve performance when data in HFiles much more than
> BlockChache (usual story in BigData). The idea - caching only part of DATA
> blocks. It is good becouse LruBlockCache starts to work and save huge amount
> of GC.
> Sometimes we have more data than can fit into BlockCache and it is cause a
> high rate of evictions. In this case we can skip cache a block N and insted
> cache the N+1th block. Anyway we would evict N block quite soon and that why
> that skipping good for performance.
> ---
> Some information below isn't actual
> ---
>
>
> Example:
> Imagine we have little cache, just can fit only 1 block and we are trying to
> read 3 blocks with offsets:
> 124
> 198
> 223
> Current way - we put the block 124, then put 198, evict 124, put 223, evict
> 198. A lot of work (5 actions).
> With the feature - last few digits evenly distributed from 0 to 99. When we
> divide by modulus we got:
> 124 -> 24
> 198 -> 98
> 223 -> 23
> It helps to sort them. Some part, for example below 50 (if we set
> *hbase.lru.cache.data.block.percent* = 50) go into the cache. And skip
> others. It means we will not try to handle the block 198 and save CPU for
> other job. In the result - we put block 124, then put 223, evict 124 (3
> actions).
> See the picture in attachment with test below. Requests per second is higher,
> GC is lower.
>
> The key point of the code:
> Added the parameter: *hbase.lru.cache.data.block.percent* which by default =
> 100
>
> But if we set it 1-99, then will work the next logic:
>
>
> {code:java}
> public void cacheBlo
[jira] [Comment Edited] (HBASE-23887) BlockCache performance improve by reduce eviction rate
[
https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204523#comment-17204523
]
Danil Lipovoy edited comment on HBASE-23887 at 9/30/20, 10:52 AM:
--
Thank you for your interest)
I created HBASE-25123 and maybe somebody will release possibility to set
classes L1 realisation.
was (Author: pustota):
Thank you for your interest)
But it looks like the feature will never merged because it has dragged on for 7
month and nothing happen. We are just discussing what would be good to do
(more performance tests, resolve conflicts etc) and I can't see the end of that.
I don't understand how open source development works, sometimes some created
issue like HBASE-24915 which save 1% CPU and it is apply without discussion in
1 day. But in this case although is saving huge amount of CPU but something
went wrong and I have no idea why. Maybe because HBASE-24915 has priority
"major" but I set just "minor";))
So I created HBASE-25123 and maybe somebody will release possibility to set
classes L1 realisation.
> BlockCache performance improve by reduce eviction rate
> --
>
> Key: HBASE-23887
> URL: https://issues.apache.org/jira/browse/HBASE-23887
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, Performance
>Reporter: Danil Lipovoy
>Assignee: Danil Lipovoy
>Priority: Minor
> Attachments: 1582787018434_rs_metrics.jpg,
> 1582801838065_rs_metrics_new.png, BC_LongRun.png,
> BlockCacheEvictionProcess.gif, BlockCacheEvictionProcess.gif, cmp.png,
> evict_BC100_vs_BC23.png, eviction_100p.png, eviction_100p.png,
> eviction_100p.png, gc_100p.png, graph.png, image-2020-06-07-08-11-11-929.png,
> image-2020-06-07-08-19-00-922.png, image-2020-06-07-12-07-24-903.png,
> image-2020-06-07-12-07-30-307.png, image-2020-06-08-17-38-45-159.png,
> image-2020-06-08-17-38-52-579.png, image-2020-06-08-18-35-48-366.png,
> image-2020-06-14-20-51-11-905.png, image-2020-06-22-05-57-45-578.png,
> image-2020-09-23-09-48-59-714.png, image-2020-09-23-10-06-11-189.png,
> ratio.png, ratio2.png, read_requests_100pBC_vs_23pBC.png, requests_100p.png,
> requests_100p.png, requests_new2_100p.png, requests_new_100p.png, scan.png,
> scan_and_gets.png, scan_and_gets2.png, wave.png, ycsb_logs.zip
>
>
> Hi!
> I first time here, correct me please if something wrong.
> All latest information is here:
> [https://docs.google.com/document/d/1X8jVnK_3lp9ibpX6lnISf_He-6xrHZL0jQQ7hoTV0-g/edit?usp=sharing]
> I want propose how to improve performance when data in HFiles much more than
> BlockChache (usual story in BigData). The idea - caching only part of DATA
> blocks. It is good becouse LruBlockCache starts to work and save huge amount
> of GC.
> Sometimes we have more data than can fit into BlockCache and it is cause a
> high rate of evictions. In this case we can skip cache a block N and insted
> cache the N+1th block. Anyway we would evict N block quite soon and that why
> that skipping good for performance.
> ---
> Some information below isn't actual
> ---
>
>
> Example:
> Imagine we have little cache, just can fit only 1 block and we are trying to
> read 3 blocks with offsets:
> 124
> 198
> 223
> Current way - we put the block 124, then put 198, evict 124, put 223, evict
> 198. A lot of work (5 actions).
> With the feature - last few digits evenly distributed from 0 to 99. When we
> divide by modulus we got:
> 124 -> 24
> 198 -> 98
> 223 -> 23
> It helps to sort them. Some part, for example below 50 (if we set
> *hbase.lru.cache.data.block.percent* = 50) go into the cache. And skip
> others. It means we will not try to handle the block 198 and save CPU for
> other job. In the result - we put block 124, then put 223, evict 124 (3
> actions).
> See the picture in attachment with test below. Requests per second is higher,
> GC is lower.
>
> The key point of the code:
> Added the parameter: *hbase.lru.cache.data.block.percent* which by default =
> 100
>
> But if we set it 1-99, then will work the next logic:
>
>
> {code:java}
> public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean
> inMemory) {
> if (cacheDataBlockPercent != 100 && buf.getBlockType().isData())
> if (cacheKey.getOffset() % 100 >= cacheDataBlockPercent)
> return;
> ...
> // the same code as usual
> }
> {code}
>
> Other parameters help to control when this logic will be enabled. It means it
> will work only while heavy reading going on.
> hbase.lru.cache.heavy.eviction.count.limit - set how many times have to run
> eviction process that start to avoid of putting data to BlockCache
> hbase.lru.cache.heavy.eviction.bytes.size.limit - set how many bytes have to
>
[GitHub] [hbase] Reidddddd merged pull request #2472: HBASE-25122 [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory
Reidd merged pull request #2472: URL: https://github.com/apache/hbase/pull/2472 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25122) [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory
[ https://issues.apache.org/jira/browse/HBASE-25122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25122: -- Component/s: test > [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory > --- > > Key: HBASE-25122 > URL: https://issues.apache.org/jira/browse/HBASE-25122 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25122) [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory
[ https://issues.apache.org/jira/browse/HBASE-25122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan resolved HBASE-25122. --- Hadoop Flags: Reviewed Resolution: Fixed > [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory > --- > > Key: HBASE-25122 > URL: https://issues.apache.org/jira/browse/HBASE-25122 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Reidddddd commented on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll
Reidd commented on pull request #2194: URL: https://github.com/apache/hbase/pull/2194#issuecomment-701321474 I read @anoopsjohn's comment. If we can make sure the `FindBugs` is fine here, please add `SuppressWarnings` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] nyl3532016 commented on a change in pull request #2473: HBASE-25115 HFilePrettyPrinter can't seek to the row which is the first row of a hfile
nyl3532016 commented on a change in pull request #2473:
URL: https://github.com/apache/hbase/pull/2473#discussion_r497395411
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java
##
@@ -323,15 +323,20 @@ public int processFile(Path file, boolean checkRootDir)
throws IOException {
HFileScanner scanner = reader.getScanner(false, false, false);
fileStats = new KeyValueStatsCollector();
boolean shouldScanKeysValues = false;
+ boolean needSeekToBegin = true;
if (this.isSeekToRow) {
// seek to the first kv on this row
-shouldScanKeysValues =
- (scanner.seekTo(PrivateCellUtil.createFirstOnRow(this.row)) != -1);
- } else {
+if (scanner.seekTo(PrivateCellUtil.createFirstOnRow(this.row)) != -1) {
+ needSeekToBegin = false;
+ shouldScanKeysValues = true;
+}
+ }
+ if (needSeekToBegin) {
Review comment:
Yes, but only when the row is the first row in hfile,
`scanner.seekTo(PrivateCellUtil.createFirstOnRow(this.row))` return -1
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2478: HBASE-24503 : Backport HBASE-24492 Remove infinite loop from ProtobufLogReader#readNext (#1831)
Apache-HBase commented on pull request #2478: URL: https://github.com/apache/hbase/pull/2478#issuecomment-701336050 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 42s | branch-2.3 passed | | +1 :green_heart: | compile | 1m 14s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 6m 21s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 46s | hbase-server in branch-2.3 failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 18s | the patch passed | | +1 :green_heart: | compile | 1m 14s | the patch passed | | +1 :green_heart: | javac | 1m 14s | the patch passed | | +1 :green_heart: | shadedjars | 6m 18s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 43s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 136m 4s | hbase-server in the patch passed. | | | | 164m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2478/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2478 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 380887d539a2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / c1649a9d51 | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2478/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2478/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2478/1/testReport/ | | Max. process+thread count | 3822 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2478/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2472: HBASE-25122 [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory
Apache-HBase commented on pull request #2472: URL: https://github.com/apache/hbase/pull/2472#issuecomment-701336895 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 10m 20s | branch-1 passed | | +1 :green_heart: | compile | 0m 54s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | compile | 0m 53s | branch-1 passed with JDK v1.7.0_272 | | +1 :green_heart: | checkstyle | 2m 7s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 45s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 0s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 52s | branch-1 passed with JDK v1.7.0_272 | | +0 :ok: | spotbugs | 3m 50s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 46s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 28s | the patch passed | | +1 :green_heart: | compile | 0m 53s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javac | 0m 53s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | javac | 0m 55s | the patch passed | | +1 :green_heart: | checkstyle | 1m 57s | hbase-server: The patch generated 0 new + 3 unchanged - 3 fixed = 3 total (was 6) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 5m 54s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 43s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 52s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | findbugs | 4m 35s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 165m 45s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 54s | The patch does not generate ASF License warnings. | | | | 217m 12s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestReplicasClient | | | hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2472/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2472 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 55d15ca664bb 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2472/out/precommit/personality/provided.sh | | git revision | branch-1 / e76e7c9 | | Default Java | 1.7.0_272 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2472/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2472/2/testReport/ | | Max. process+thread count | 3645 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2472/2/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25115) HFilePrettyPrinter can't seek to the row which is the first row of a hfile
[
https://issues.apache.org/jira/browse/HBASE-25115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204669#comment-17204669
]
niuyulin commented on HBASE-25115:
--
[~zcq_rambo] Yes, your code is simplified, I will modify my PR on master branch
> HFilePrettyPrinter can't seek to the row which is the first row of a hfile
> --
>
> Key: HBASE-25115
> URL: https://issues.apache.org/jira/browse/HBASE-25115
> Project: HBase
> Issue Type: Bug
> Components: HFile, tooling
>Affects Versions: 2.2.6
>Reporter: Chaoqiang Zhong
>Assignee: niuyulin
>Priority: Major
> Attachments: HBASE-25115.patch
>
>
> This issue can be reproduced by below steps:
> * make a hfile contains two rows '000' and '001';
> {code:java}
> D:\bin>hbase hfile -p -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> K: 000/f1:a/1601282789548/Put/vlen=4/seqid=4 V:
> K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5 V:
> Scanned kv count -> 2{code}
> * '001' can be seeked to;
> {code:java}
> D:\bin>hbase hfile -e -w 001 -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5
> Scanned kv count -> 1{code}
> * but '000' can't be seeked to;
> {code:java}
> D:\bin>hbase hfile -e -w 000 -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> Scanned kv count -> 0{code}
> In HFilePrettyPrinter we use
> "scanner.seekTo(PrivateCellUtil.createFirstOnRow(this.row))" to seek to
> row.But this method will retrurn -1 when the row is the first row of hfile.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2480: HBASE-25034:Table regions details on master GUI display slowly.
Apache-HBase commented on pull request #2480: URL: https://github.com/apache/hbase/pull/2480#issuecomment-701354121 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 3s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 36s | branch-2.2 passed | | +1 :green_heart: | javadoc | 0m 40s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 5s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 175m 37s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 39s | The patch does not generate ASF License warnings. | | | | 187m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2480/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2480 | | JIRA Issue | HBASE-25034 | | Optional Tests | dupname asflicense javac javadoc unit | | uname | Linux 3ba6289dbe96 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-2480/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 4bf490c867 | | Default Java | 1.8.0_181 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2480/1/testReport/ | | Max. process+thread count | 3395 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2480/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2482: HBASE-25034:Table regions details on master GUI display slowly.
Apache-HBase commented on pull request #2482: URL: https://github.com/apache/hbase/pull/2482#issuecomment-701363715 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 17s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 23s | branch-2.2 passed | | +1 :green_heart: | javadoc | 0m 39s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 6s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 175m 7s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 30s | The patch does not generate ASF License warnings. | | | | 186m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2482/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2482 | | JIRA Issue | HBASE-25034 | | Optional Tests | dupname asflicense javac javadoc unit | | uname | Linux 6b939ec92f31 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2482/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 4bf490c867 | | Default Java | 1.8.0_181 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2482/1/testReport/ | | Max. process+thread count | 3194 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2482/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2475: HBASE-25034:Table regions details on master GUI display slowly.
Apache-HBase commented on pull request #2475: URL: https://github.com/apache/hbase/pull/2475#issuecomment-701365990 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 22s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 8s | branch-2.2 passed | | +1 :green_heart: | javadoc | 0m 44s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | javadoc | 0m 43s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 296m 37s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 39s | The patch does not generate ASF License warnings. | | | | 308m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2475/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2475 | | JIRA Issue | HBASE-25034 | | Optional Tests | dupname asflicense javac javadoc unit | | uname | Linux 477e6b9400cd 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2475/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 4bf490c867 | | Default Java | 1.8.0_181 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2475/1/testReport/ | | Max. process+thread count | 2893 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2475/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-701385393 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 40s | master passed | | +1 :green_heart: | checkstyle | 1m 19s | master passed | | +1 :green_heart: | spotbugs | 2m 32s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 1m 33s | root in the patch failed. | | +1 :green_heart: | checkstyle | 1m 12s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | -1 :x: | hadoopcheck | 1m 36s | The patch causes 20 errors with Hadoop v3.1.2. | | -1 :x: | hadoopcheck | 3m 16s | The patch causes 20 errors with Hadoop v3.2.1. | | -1 :x: | spotbugs | 0m 17s | hbase-procedure in the patch failed. | | -1 :x: | spotbugs | 0m 34s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. | | | | 17m 16s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 802294cc2d21 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-general-check/output/patch-mvninstall-root.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-general-check/output/patch-javac-3.1.2.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-general-check/output/patch-javac-3.2.1.txt | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-general-check/output/patch-spotbugs-hbase-procedure.txt | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-general-check/output/patch-spotbugs-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sanjeetnishad95 commented on a change in pull request #2353: HBASE-24981 Enable table replication fails from 1.x to 2.x if table a…
sanjeetnishad95 commented on a change in pull request #2353: URL: https://github.com/apache/hbase/pull/2353#discussion_r497504459 ## File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/ColumnFamilyDescriptorBuilder.java ## @@ -294,16 +294,18 @@ DEFAULT_VALUES.put(BLOCKCACHE, String.valueOf(DEFAULT_BLOCKCACHE)); DEFAULT_VALUES.put(KEEP_DELETED_CELLS, String.valueOf(DEFAULT_KEEP_DELETED)); DEFAULT_VALUES.put(DATA_BLOCK_ENCODING, String.valueOf(DEFAULT_DATA_BLOCK_ENCODING)); -DEFAULT_VALUES.put(CACHE_DATA_ON_WRITE, String.valueOf(DEFAULT_CACHE_DATA_ON_WRITE)); -DEFAULT_VALUES.put(CACHE_INDEX_ON_WRITE, String.valueOf(DEFAULT_CACHE_INDEX_ON_WRITE)); -DEFAULT_VALUES.put(CACHE_BLOOMS_ON_WRITE, String.valueOf(DEFAULT_CACHE_BLOOMS_ON_WRITE)); -DEFAULT_VALUES.put(EVICT_BLOCKS_ON_CLOSE, String.valueOf(DEFAULT_EVICT_BLOCKS_ON_CLOSE)); -DEFAULT_VALUES.put(PREFETCH_BLOCKS_ON_OPEN, String.valueOf(DEFAULT_PREFETCH_BLOCKS_ON_OPEN)); // Do NOT add this key/value by default. NEW_VERSION_BEHAVIOR is NOT defined in hbase1 so // it is not possible to make an hbase1 HCD the same as an hbase2 HCD and so the replication // compare of schemas will fail. It is OK not adding the below to the initial map because of // fetch of this value, we will check for null and if null will return the default. // DEFAULT_VALUES.put(NEW_VERSION_BEHAVIOR, String.valueOf(DEFAULT_NEW_VERSION_BEHAVIOR)); + +// DEFAULT_VALUES.put(CACHE_DATA_ON_WRITE, String.valueOf(DEFAULT_CACHE_DATA_ON_WRITE)); +// DEFAULT_VALUES.put(CACHE_INDEX_ON_WRITE, String.valueOf(DEFAULT_CACHE_INDEX_ON_WRITE)); +// DEFAULT_VALUES.put(CACHE_BLOOMS_ON_WRITE, String.valueOf(DEFAULT_CACHE_BLOOMS_ON_WRITE)); +// DEFAULT_VALUES.put(EVICT_BLOCKS_ON_CLOSE, String.valueOf(DEFAULT_EVICT_BLOCKS_ON_CLOSE)); +// DEFAULT_VALUES.put(PREFETCH_BLOCKS_ON_OPEN, String.valueOf(DEFAULT_PREFETCH_BLOCKS_ON_OPEN)); + Review comment: @saintstack @virajjasani are the changes ok? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sanjeetnishad95 commented on a change in pull request #2353: HBASE-24981 Enable table replication fails from 1.x to 2.x if table a…
sanjeetnishad95 commented on a change in pull request #2353: URL: https://github.com/apache/hbase/pull/2353#discussion_r497504459 ## File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/ColumnFamilyDescriptorBuilder.java ## @@ -294,16 +294,18 @@ DEFAULT_VALUES.put(BLOCKCACHE, String.valueOf(DEFAULT_BLOCKCACHE)); DEFAULT_VALUES.put(KEEP_DELETED_CELLS, String.valueOf(DEFAULT_KEEP_DELETED)); DEFAULT_VALUES.put(DATA_BLOCK_ENCODING, String.valueOf(DEFAULT_DATA_BLOCK_ENCODING)); -DEFAULT_VALUES.put(CACHE_DATA_ON_WRITE, String.valueOf(DEFAULT_CACHE_DATA_ON_WRITE)); -DEFAULT_VALUES.put(CACHE_INDEX_ON_WRITE, String.valueOf(DEFAULT_CACHE_INDEX_ON_WRITE)); -DEFAULT_VALUES.put(CACHE_BLOOMS_ON_WRITE, String.valueOf(DEFAULT_CACHE_BLOOMS_ON_WRITE)); -DEFAULT_VALUES.put(EVICT_BLOCKS_ON_CLOSE, String.valueOf(DEFAULT_EVICT_BLOCKS_ON_CLOSE)); -DEFAULT_VALUES.put(PREFETCH_BLOCKS_ON_OPEN, String.valueOf(DEFAULT_PREFETCH_BLOCKS_ON_OPEN)); // Do NOT add this key/value by default. NEW_VERSION_BEHAVIOR is NOT defined in hbase1 so // it is not possible to make an hbase1 HCD the same as an hbase2 HCD and so the replication // compare of schemas will fail. It is OK not adding the below to the initial map because of // fetch of this value, we will check for null and if null will return the default. // DEFAULT_VALUES.put(NEW_VERSION_BEHAVIOR, String.valueOf(DEFAULT_NEW_VERSION_BEHAVIOR)); + +// DEFAULT_VALUES.put(CACHE_DATA_ON_WRITE, String.valueOf(DEFAULT_CACHE_DATA_ON_WRITE)); +// DEFAULT_VALUES.put(CACHE_INDEX_ON_WRITE, String.valueOf(DEFAULT_CACHE_INDEX_ON_WRITE)); +// DEFAULT_VALUES.put(CACHE_BLOOMS_ON_WRITE, String.valueOf(DEFAULT_CACHE_BLOOMS_ON_WRITE)); +// DEFAULT_VALUES.put(EVICT_BLOCKS_ON_CLOSE, String.valueOf(DEFAULT_EVICT_BLOCKS_ON_CLOSE)); +// DEFAULT_VALUES.put(PREFETCH_BLOCKS_ON_OPEN, String.valueOf(DEFAULT_PREFETCH_BLOCKS_ON_OPEN)); + Review comment: @saintstack are the changes ok? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-701390058 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 5s | master passed | | +1 :green_heart: | compile | 1m 28s | master passed | | +1 :green_heart: | shadedjars | 6m 41s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 19s | hbase-procedure in master failed. | | -0 :warning: | javadoc | 0m 40s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 1m 55s | root in the patch failed. | | -1 :x: | compile | 0m 18s | hbase-procedure in the patch failed. | | -1 :x: | compile | 0m 40s | hbase-server in the patch failed. | | -0 :warning: | javac | 0m 18s | hbase-procedure in the patch failed. | | -0 :warning: | javac | 0m 40s | hbase-server in the patch failed. | | -1 :x: | shadedjars | 5m 5s | patch has 20 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 22s | hbase-procedure in the patch failed. | | -0 :warning: | javadoc | 0m 26s | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 0m 22s | hbase-procedure in the patch failed. | | -1 :x: | unit | 0m 44s | hbase-server in the patch failed. | | | | 25m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3b4d0ad08dff 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-procedure.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk11-hadoop3-check/output/patch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-procedure.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-procedure.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-procedure.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-procedure.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/testReport/ | | Max. process+thread count | 99 (vs. ulimit of 3) | | modules
[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-701389861 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 53s | master passed | | +1 :green_heart: | compile | 1m 22s | master passed | | +1 :green_heart: | shadedjars | 6m 58s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 1m 46s | root in the patch failed. | | -1 :x: | compile | 0m 19s | hbase-procedure in the patch failed. | | -1 :x: | compile | 0m 39s | hbase-server in the patch failed. | | -0 :warning: | javac | 0m 19s | hbase-procedure in the patch failed. | | -0 :warning: | javac | 0m 39s | hbase-server in the patch failed. | | -1 :x: | shadedjars | 4m 29s | patch has 20 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 38s | hbase-server generated 2 new + 29 unchanged - 0 fixed = 31 total (was 29) | ||| _ Other Tests _ | | -1 :x: | unit | 0m 22s | hbase-procedure in the patch failed. | | -1 :x: | unit | 0m 41s | hbase-server in the patch failed. | | | | 24m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 79ccdd3b43ae 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Default Java | 1.8.0_232 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk8-hadoop3-check/output/patch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-procedure.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-procedure.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-procedure.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/testReport/ | | Max. process+thread count | 88 (vs. ulimit of 3) | | modules | C: hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apa
[GitHub] [hbase] Apache-HBase commented on pull request #2476: HBASE-25121 Refactor MetaTableAccessor.addRegionsToMeta and its usage…
Apache-HBase commented on pull request #2476: URL: https://github.com/apache/hbase/pull/2476#issuecomment-701396027 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 10m 10s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 17s | master passed | | +1 :green_heart: | compile | 2m 20s | master passed | | +1 :green_heart: | shadedjars | 8m 44s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 23s | hbase-balancer in master failed. | | -0 :warning: | javadoc | 0m 39s | hbase-client in master failed. | | -0 :warning: | javadoc | 1m 2s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 6m 6s | the patch passed | | +1 :green_heart: | compile | 2m 31s | the patch passed | | +1 :green_heart: | javac | 2m 31s | the patch passed | | +1 :green_heart: | shadedjars | 8m 39s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 31s | hbase-client in the patch failed. | | -0 :warning: | javadoc | 0m 18s | hbase-balancer in the patch failed. | | -0 :warning: | javadoc | 0m 52s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 39s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 0m 26s | hbase-balancer in the patch passed. | | +1 :green_heart: | unit | 254m 32s | hbase-server in the patch passed. | | | | 307m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2476 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 2743c3cae69f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-balancer.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-balancer.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/1/testReport/ | | Max. process+thread count | 3184 (vs. ulimit of 3) | | modules | C: hbase-client hbase-balancer hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25099) Change meta replica count by altering meta table descriptor
[ https://issues.apache.org/jira/browse/HBASE-25099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-25099: -- Summary: Change meta replica count by altering meta table descriptor (was: Change meta replica number by altering meta table descriptor) > Change meta replica count by altering meta table descriptor > --- > > Key: HBASE-25099 > URL: https://issues.apache.org/jira/browse/HBASE-25099 > Project: HBase > Issue Type: Improvement > Components: meta, read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > As now we support altering meta table, it will be better to also deal with > changing meta replica number using altering meta, i.e, we could unify the > logic in MasterMetaBootstrap to ModifyTableProcedure, and another benefit is > that we do not need to restart master when changing the replica number for > meta. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2473: HBASE-25115 HFilePrettyPrinter can't seek to the row which is the first row of a hfile
Apache-HBase commented on pull request #2473: URL: https://github.com/apache/hbase/pull/2473#issuecomment-701403081 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 2s | master passed | | +1 :green_heart: | checkstyle | 1m 57s | master passed | | +1 :green_heart: | spotbugs | 3m 35s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 44s | the patch passed | | +1 :green_heart: | checkstyle | 1m 54s | hbase-server: The patch generated 0 new + 10 unchanged - 1 fixed = 10 total (was 11) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 59s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 59s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 52m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2473 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 0595f25f9d3f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2478: HBASE-24503 : Backport HBASE-24492 Remove infinite loop from ProtobufLogReader#readNext (#1831)
Apache-HBase commented on pull request #2478: URL: https://github.com/apache/hbase/pull/2478#issuecomment-701406170 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 7s | branch-2.3 passed | | +1 :green_heart: | compile | 1m 17s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 6m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 46s | the patch passed | | +1 :green_heart: | compile | 1m 21s | the patch passed | | +1 :green_heart: | javac | 1m 21s | the patch passed | | +1 :green_heart: | shadedjars | 8m 40s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 55s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 265m 36s | hbase-server in the patch passed. | | | | 298m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2478/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2478 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 11e6b95decad 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / c1649a9d51 | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2478/1/testReport/ | | Max. process+thread count | 2457 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2478/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2477: HBASE-25034:Table regions details on master GUI display slowly.
Apache-HBase commented on pull request #2477: URL: https://github.com/apache/hbase/pull/2477#issuecomment-701411884 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 26s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 19s | branch-2.2 passed | | +1 :green_heart: | javadoc | 0m 49s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 9s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | javadoc | 0m 47s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 300m 7s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 31s | The patch does not generate ASF License warnings. | | | | 313m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2477/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2477 | | JIRA Issue | HBASE-25034 | | Optional Tests | dupname asflicense javac javadoc unit | | uname | Linux dcf7086fb27a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-2477/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 4bf490c867 | | Default Java | 1.8.0_181 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2477/1/testReport/ | | Max. process+thread count | 2492 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2477/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2476: HBASE-25121 Refactor MetaTableAccessor.addRegionsToMeta and its usage…
Apache-HBase commented on pull request #2476: URL: https://github.com/apache/hbase/pull/2476#issuecomment-701412306 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 41s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 23s | master passed | | +1 :green_heart: | compile | 2m 32s | master passed | | +1 :green_heart: | shadedjars | 10m 4s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 0s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 34s | the patch passed | | +1 :green_heart: | compile | 2m 20s | the patch passed | | +1 :green_heart: | javac | 2m 20s | the patch passed | | +1 :green_heart: | shadedjars | 9m 26s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 29s | hbase-client in the patch passed. | | +1 :green_heart: | javadoc | 0m 18s | hbase-balancer generated 0 new + 1 unchanged - 1 fixed = 1 total (was 2) | | +1 :green_heart: | javadoc | 0m 48s | hbase-server in the patch passed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 38s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 0m 28s | hbase-balancer in the patch passed. | | -1 :x: | unit | 286m 36s | hbase-server in the patch failed. | | | | 334m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2476 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b147edf5c174 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Default Java | 1.8.0_232 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/1/testReport/ | | Max. process+thread count | 2704 (vs. ulimit of 3) | | modules | C: hbase-client hbase-balancer hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25099) Change meta replica count by altering meta table descriptor
[ https://issues.apache.org/jira/browse/HBASE-25099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204751#comment-17204751 ] Duo Zhang commented on HBASE-25099: --- Attached a design doc. > Change meta replica count by altering meta table descriptor > --- > > Key: HBASE-25099 > URL: https://issues.apache.org/jira/browse/HBASE-25099 > Project: HBase > Issue Type: Improvement > Components: meta, read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > As now we support altering meta table, it will be better to also deal with > changing meta replica number using altering meta, i.e, we could unify the > logic in MasterMetaBootstrap to ModifyTableProcedure, and another benefit is > that we do not need to restart master when changing the replica number for > meta. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25124) Support changing meta replica count without disabling table
Duo Zhang created HBASE-25124: - Summary: Support changing meta replica count without disabling table Key: HBASE-25124 URL: https://issues.apache.org/jira/browse/HBASE-25124 Project: HBase Issue Type: Sub-task Components: meta, proc-v2 Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-25124) Support changing meta replica count without disabling table
[ https://issues.apache.org/jira/browse/HBASE-25124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-25124: - Assignee: Duo Zhang > Support changing meta replica count without disabling table > --- > > Key: HBASE-25124 > URL: https://issues.apache.org/jira/browse/HBASE-25124 > Project: HBase > Issue Type: Sub-task > Components: meta, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24764) Add support of adding base peer configs via hbase-site.xml for all replication peers.
[
https://issues.apache.org/jira/browse/HBASE-24764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204784#comment-17204784
]
Hudson commented on HBASE-24764:
Results for branch branch-1
[build #34 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/34/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/34//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/34//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/34//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Add support of adding base peer configs via hbase-site.xml for all
> replication peers.
> -
>
> Key: HBASE-24764
> URL: https://issues.apache.org/jira/browse/HBASE-24764
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 3.0.0-alpha-1, 1.7.0, 2.4.0
>Reporter: Ankit Jain
>Assignee: Ankit Jain
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0
>
> Attachments: HBASE-24764-master.patch
>
>
> Today, if a user needs to apply some common base peer configs to all the
> replication peers on a given cluster, the only way is to execute
> update_peer_config via CLI which requires manual intervention and can be
> tedious in case of large deployment fleet.
> As part of this JIRA, we plan to add the support to have base replication
> peer configs as part of hbase-site.xml like
> hbase.replication.peer.base.config="k1=v1;k2=v2.." which can be easily
> updated and applied as part of a rolling restart. Example below:
>
> hbase.replication.peer.base.configs
>
> hbase.replication.source.custom.walentryfilters=x,y,z;hbase.rpc.protection=abc;hbase.xxx.custom_property=123
>
> This will be empty by default, but one can override to have base configs in
> place.
> The final peer configuration would be a merge of this newly added base config
> + whatever users override during the peer creation/update (if any).
> Related Jira: https://issues.apache.org/jira/browse/HBASE-17543. HBASE-17543
> added the support to add the WALEntryFilters to default endpoint via peer
> configuration. By this new Jira we are extending the support to update peer
> configs via hbase-site.xml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25114) [Flake Test][branch-1] TestFromClientSide#testCacheOnWriteEvictOnClose
[
https://issues.apache.org/jira/browse/HBASE-25114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204782#comment-17204782
]
Hudson commented on HBASE-25114:
Results for branch branch-1
[build #34 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/34/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/34//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/34//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/34//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> [Flake Test][branch-1] TestFromClientSide#testCacheOnWriteEvictOnClose
> --
>
> Key: HBASE-25114
> URL: https://issues.apache.org/jira/browse/HBASE-25114
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 1.7.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25119) Fix property name added as part of HBASE-24764 in branch-1
[
https://issues.apache.org/jira/browse/HBASE-25119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204783#comment-17204783
]
Hudson commented on HBASE-25119:
Results for branch branch-1
[build #34 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/34/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/34//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/34//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/34//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Fix property name added as part of HBASE-24764 in branch-1
> --
>
> Key: HBASE-25119
> URL: https://issues.apache.org/jira/browse/HBASE-25119
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Reporter: Ankit Jain
>Assignee: Ankit Jain
>Priority: Minor
> Fix For: 1.7.0
>
>
> A new property was added as part of this
> https://issues.apache.org/jira/browse/HBASE-24764.
> Master branch has the name as "hbase.replication.peer.base.config"
> while branch-1 has the name as "hbase.replication.peer.default.config"
>
> As part of this Jira fix the name in branch-1 to be consistent.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25122) [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory
[
https://issues.apache.org/jira/browse/HBASE-25122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204785#comment-17204785
]
Hudson commented on HBASE-25122:
Results for branch branch-1
[build #34 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/34/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/34//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/34//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/34//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory
> ---
>
> Key: HBASE-25122
> URL: https://issues.apache.org/jira/browse/HBASE-25122
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 1.7.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2476: HBASE-25121 Refactor MetaTableAccessor.addRegionsToMeta and its usage…
Apache-HBase commented on pull request #2476: URL: https://github.com/apache/hbase/pull/2476#issuecomment-701460355 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 1s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 41s | master passed | | +1 :green_heart: | checkstyle | 2m 10s | master passed | | +1 :green_heart: | spotbugs | 4m 41s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 32s | the patch passed | | +1 :green_heart: | checkstyle | 0m 32s | The patch passed checkstyle in hbase-client | | +1 :green_heart: | checkstyle | 0m 14s | The patch passed checkstyle in hbase-balancer | | +1 :green_heart: | checkstyle | 1m 24s | hbase-server: The patch generated 0 new + 158 unchanged - 9 fixed = 158 total (was 167) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 14m 18s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 5m 28s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 39s | The patch does not generate ASF License warnings. | | | | 49m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2476 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 1f3a9095b401 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-client hbase-balancer hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2476/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25099) Change meta replica count by altering meta table descriptor
[ https://issues.apache.org/jira/browse/HBASE-25099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204804#comment-17204804 ] Michael Stack commented on HBASE-25099: --- Yeah, I think #2. You made good argument when I wanted to disable meta to allow generally altering schema (back when I wanted to add indices and blooms to the hbase:meta table). > Change meta replica count by altering meta table descriptor > --- > > Key: HBASE-25099 > URL: https://issues.apache.org/jira/browse/HBASE-25099 > Project: HBase > Issue Type: Improvement > Components: meta, read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > As now we support altering meta table, it will be better to also deal with > changing meta replica number using altering meta, i.e, we could unify the > logic in MasterMetaBootstrap to ModifyTableProcedure, and another benefit is > that we do not need to restart master when changing the replica number for > meta. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2479: HBASE-24503 : Backport HBASE-24492 Remove infinite loop from ProtobufLogReader#readNext (#1831)
Apache-HBase commented on pull request #2479: URL: https://github.com/apache/hbase/pull/2479#issuecomment-701480715 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 44s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 28s | branch-2.2 passed | | +1 :green_heart: | compile | 1m 8s | branch-2.2 passed | | +1 :green_heart: | checkstyle | 1m 45s | branch-2.2 passed | | +1 :green_heart: | shadedjars | 5m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 47s | branch-2.2 passed | | +0 :ok: | spotbugs | 4m 28s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 25s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 7s | the patch passed | | +1 :green_heart: | compile | 1m 11s | the patch passed | | +1 :green_heart: | javac | 1m 11s | the patch passed | | +1 :green_heart: | checkstyle | 1m 35s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 40s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 33m 50s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | | +1 :green_heart: | findbugs | 3m 12s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 331m 58s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 33s | The patch does not generate ASF License warnings. | | | | 414m 14s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestSnapshotCloneIndependence | | | hadoop.hbase.master.TestDLSFSHLog | | | hadoop.hbase.master.TestAssignmentManagerMetrics | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2479/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2479 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 0d707f663a46 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2479/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 4bf490c867 | | Default Java | 1.8.0_181 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2479/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2479/1/testReport/ | | Max. process+thread count | 2623 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2479/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2473: HBASE-25115 HFilePrettyPrinter can't seek to the row which is the first row of a hfile
Apache-HBase commented on pull request #2473: URL: https://github.com/apache/hbase/pull/2473#issuecomment-701481394 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 6s | master passed | | +1 :green_heart: | compile | 1m 31s | master passed | | +1 :green_heart: | shadedjars | 9m 50s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 59s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 1s | the patch passed | | +1 :green_heart: | compile | 2m 6s | the patch passed | | +1 :green_heart: | javac | 2m 6s | the patch passed | | +1 :green_heart: | shadedjars | 7m 37s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 43s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 141m 29s | hbase-server in the patch passed. | | | | 178m 18s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2473 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7fc4e2c8ad15 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/2/testReport/ | | Max. process+thread count | 3664 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2473: HBASE-25115 HFilePrettyPrinter can't seek to the row which is the first row of a hfile
Apache-HBase commented on pull request #2473: URL: https://github.com/apache/hbase/pull/2473#issuecomment-701488275 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 8s | master passed | | +1 :green_heart: | compile | 1m 3s | master passed | | +1 :green_heart: | shadedjars | 7m 57s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 3s | the patch passed | | +1 :green_heart: | compile | 1m 6s | the patch passed | | +1 :green_heart: | javac | 1m 6s | the patch passed | | +1 :green_heart: | shadedjars | 7m 59s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 157m 39s | hbase-server in the patch passed. | | | | 189m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2473 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux e56689668fe1 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/2/testReport/ | | Max. process+thread count | 4198 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2473/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2453: HBASE-24813 ReplicationSource should clear buffer usage on Replicatio…
Apache-HBase commented on pull request #2453: URL: https://github.com/apache/hbase/pull/2453#issuecomment-701488699 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 50s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 22s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 37s | branch-2 passed | | +1 :green_heart: | spotbugs | 2m 58s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 2s | the patch passed | | -0 :warning: | checkstyle | 1m 30s | hbase-server: The patch generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 15m 31s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 20s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 43m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2453/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2453 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux b97f83bd3d8c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / f973a8087f | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2453/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 95 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2453/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2468: HBASE-25109 Add MR Counters to WALPlayer; currently hard to tell if i…
saintstack commented on pull request #2468: URL: https://github.com/apache/hbase/pull/2468#issuecomment-701503995 Hm. The unit tests that are failing fail w/o this patch and they are unrelated. Let me push this and then look at the failures. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack merged pull request #2468: HBASE-25109 Add MR Counters to WALPlayer; currently hard to tell if i…
saintstack merged pull request #2468: URL: https://github.com/apache/hbase/pull/2468 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #2353: HBASE-24981 Enable table replication fails from 1.x to 2.x if table a…
virajjasani commented on pull request #2353: URL: https://github.com/apache/hbase/pull/2353#issuecomment-701506184 Planning to commit this in ~1 day unless there are any concerns. This should be applicable to all branch-2.x. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #2353: HBASE-24981 Enable table replication fails from 1.x to 2.x if table a…
virajjasani commented on pull request #2353: URL: https://github.com/apache/hbase/pull/2353#issuecomment-701506746 @saintstack 's concern is also addressed in the latest patch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-25109) Add MR Counters to WALPlayer; currently hard to tell if it is doing anything
[
https://issues.apache.org/jira/browse/HBASE-25109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-25109.
---
Fix Version/s: 2.4.0
2.3.3
3.0.0-alpha-1
Hadoop Flags: Reviewed
Assignee: Michael Stack
Resolution: Fixed
Pushed to branch-2.3+. Thanks for review [~huaxiangsun]
> Add MR Counters to WALPlayer; currently hard to tell if it is doing anything
>
>
> Key: HBASE-25109
> URL: https://issues.apache.org/jira/browse/HBASE-25109
> Project: HBase
> Issue Type: Improvement
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0
>
>
> For example, when WALPlayer runs, it emits this:
> {code:java}
> 020-09-28 11:16:05,489 INFO [LocalJobRunner Map Task Executor #0]
> mapred.Task: Final Counters for attempt_local1916643172_0001_m_00_0:
> Counters: 20
> File System Counters
> FILE: Number of bytes read=268891453
> FILE: Number of bytes written=1018719
> FILE: Number of read operations=0
> FILE: Number of large read operations=0
> FILE: Number of write operations=0
> Map-Reduce Framework
> Map input records=4375
> Map output records=5369
> Input split bytes=245
> Spilled Records=0
> Failed Shuffles=0
> Merged Map outputs=0
> GC time elapsed (ms)=59
> Total committed heap usage (bytes)=518979584
> File Input Format Counters
> Bytes Read=0
> File Output Format Counters
> Bytes Written=0 {code}
> Change it so it does this:
> {code:java}
> 020-09-28 11:16:05,489 INFO [LocalJobRunner Map Task Executor #0]
> mapred.Task: Final Counters for attempt_local1916643172_0001_m_00_0:
> Counters: 20
> File System Counters
> FILE: Number of bytes read=268891453
> FILE: Number of bytes written=1018719
> FILE: Number of read operations=0
> FILE: Number of large read operations=0
> FILE: Number of write operations=0
> Map-Reduce Framework
> Map input records=4375
> Map output records=5369
> Input split bytes=245
> Spilled Records=0
> Failed Shuffles=0
> Merged Map outputs=0
> GC time elapsed (ms)=59
> Total committed heap usage (bytes)=518979584
> org.apache.hadoop.hbase.mapreduce.WALPlayer$Counter
> CELLS_READ=89574
> CELLS_WRITTEN=89572
> DELETES=64
> PUTS=5305
> WALEDITS=4375
> File Input Format Counters
> Bytes Read=0
> File Output Format Counters
> Bytes Written=0 {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24981) Enable table replication fails from 1.x to 2.x if table already exist at peer.
[
https://issues.apache.org/jira/browse/HBASE-24981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204880#comment-17204880
]
Viraj Jasani commented on HBASE-24981:
--
Planning to commit this in a day unless there are any concerns.
> Enable table replication fails from 1.x to 2.x if table already exist at
> peer.
> ---
>
> Key: HBASE-24981
> URL: https://issues.apache.org/jira/browse/HBASE-24981
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.2.3
>Reporter: Sanjeet Nishad
>Assignee: Sanjeet Nishad
>Priority: Major
>
> If source is 1.x and peer is 2.x and a default table is created at source and
> also at peer, default table descriptors mismatch so the enable table
> replication fails with error :
>
> {code:java}
> Table t1 exists in peer cluster 1, but the table descriptors are not same
> when compared with source cluster. Thus can not enable the table's
> replication switch.{code}
> Following are the extra attributes set by default in 2.x:
> CACHE_DATA_ON_WRITE
> CACHE_INDEX_ON_WRITE
> CACHE_BLOOMS_ON_WRITE
> EVICT_BLOCKS_ON_CLOSE
> PREFETCH_BLOCKS_ON_OPEN.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] sanjeetnishad95 commented on a change in pull request #2163: HBASE-24789 When table level quota is set and violated, pre existing …
sanjeetnishad95 commented on a change in pull request #2163:
URL: https://github.com/apache/hbase/pull/2163#discussion_r497664990
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestSpaceQuotaBasicFunctioning.java
##
@@ -257,4 +258,35 @@ public boolean evaluate() throws Exception {
.filter(k -> k.getTable().equals(tableName)).count();
Assert.assertTrue(regionSizes > 0);
}
+
+ @Test
+ public void testNSAndTableQuotaRemoveTableQuotaTableInViolation() throws
Exception {
+final String ns = testName.getMethodName();
+NamespaceDescriptor nd = NamespaceDescriptor.create(ns).build();
+TEST_UTIL.getAdmin().createNamespace(nd);
+TableName table1 = helper.createTableInNamespace(nd);
+TableName table2 = helper.createTableInNamespace(nd);
+TableName table3 = helper.createTableInNamespace(nd);
+Put put = new Put(Bytes.toBytes(("to_reject")));
+put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
Bytes.toBytes("to"),
+ Bytes.toBytes("reject"));
+helper.setQuotaLimit(nd.getName(),
SpaceViolationPolicy.NO_WRITES_COMPACTIONS, 4L);
+helper.setQuotaLimit(table2, SpaceViolationPolicy.NO_WRITES, 2L);
+helper.writeData(table2, 2L * SpaceQuotaHelperForTests.ONE_MEGABYTE);
+helper.verifyViolation(SpaceViolationPolicy.NO_WRITES, table2, put);
+helper.removeQuotaFromtable(table2);
+helper.writeData(table1, 1L * SpaceQuotaHelperForTests.ONE_MEGABYTE);
+helper.writeData(table3, 1L * SpaceQuotaHelperForTests.ONE_MEGABYTE);
+Put put1 = new Put(Bytes.toBytes(("to_reject1")));
+put1.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
Bytes.toBytes("to"),
+ Bytes.toBytes("reject"));
+helper.verifyViolation(SpaceViolationPolicy.NO_WRITES_COMPACTIONS, table2,
put1);
+try {
+ TEST_UTIL.getAdmin().majorCompact(table2);
+ fail("Expected that invoking the compaction should throw an Exception");
+} catch (DoNotRetryIOException e) {
Review comment:
Couldn't unwrap as we are throwing DNRIOE directly :
`throw new DoNotRetryIOException(
"Compactions on this region are " + "disabled due to a space
quota violation.");`
But I'll update the assertion based on the message `"disabled due to a space
quota violation.".`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2468: HBASE-25109 Add MR Counters to WALPlayer; currently hard to tell if i…
saintstack commented on pull request #2468: URL: https://github.com/apache/hbase/pull/2468#issuecomment-701526513 I did clean build of branch-2 and the tests pass locally. I'll keep an eye on this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sanjeetnishad95 commented on a change in pull request #2163: HBASE-24789 When table level quota is set and violated, pre existing …
sanjeetnishad95 commented on a change in pull request #2163:
URL: https://github.com/apache/hbase/pull/2163#discussion_r497674924
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java
##
@@ -284,6 +285,9 @@ public void delete() throws IOException {
SpaceQuotaSnapshot currSnapshotOfTable =
QuotaTableUtil.getCurrentSnapshotFromQuotaTable(masterServices.getConnection(),
table);
QuotaUtil.deleteTableQuota(masterServices.getConnection(), table);
+// Removes the tableName from the current state in QuotaObserverChore
+QuotaObserverChore quotaChore =
((HMaster)masterServices).getQuotaObserverChore();
Review comment:
@ndimiduk I didn't get this comment. Masterservices interface doesn't
have QuotaObserverChore. Please let me know if have something wrong here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] gkanade opened a new pull request #2483: HBASE-25026
gkanade opened a new pull request #2483: URL: https://github.com/apache/hbase/pull/2483 Initial PR to get the metrics that indicates full region scans. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2449: HBASE-25091 Move LogComparator from ReplicationSource to AbstractFSWA…
Apache-HBase commented on pull request #2449: URL: https://github.com/apache/hbase/pull/2449#issuecomment-701546447 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 30s | master passed | | +1 :green_heart: | checkstyle | 1m 6s | master passed | | +1 :green_heart: | spotbugs | 2m 2s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 26s | the patch passed | | +1 :green_heart: | checkstyle | 1m 2s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 14s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 33m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2449/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2449 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 22b3d4bfaa0c 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 01876071b9 | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2449/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…
saintstack commented on pull request #2451: URL: https://github.com/apache/hbase/pull/2451#issuecomment-701552064 I ran the failed tests locally and they passed. Rerunning. @Apache9 Mind taking a look sir? This the direction you suggested? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
Apache-HBase commented on pull request #2483: URL: https://github.com/apache/hbase/pull/2483#issuecomment-701552462 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 59s | master passed | | +1 :green_heart: | checkstyle | 1m 30s | master passed | | +1 :green_heart: | spotbugs | 2m 47s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 44s | the patch passed | | -0 :warning: | checkstyle | 0m 14s | hbase-hadoop-compat: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | | -0 :warning: | checkstyle | 1m 14s | hbase-server: The patch generated 7 new + 266 unchanged - 0 fixed = 273 total (was 266) | | -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | hadoopcheck | 12m 2s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 7s | the patch passed | ||| _ Other Tests _ | | -1 :x: | asflicense | 0m 29s | The patch generated 1 ASF License warnings. | | | | 38m 28s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2483 | | JIRA Issue | HBASE-25026 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 18328b30aedc 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 01876071b9 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-hadoop-compat.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | whitespace | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/1/artifact/yetus-general-check/output/whitespace-eol.txt | | asflicense | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/1/artifact/yetus-general-check/output/patch-asflicense-problems.txt | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25125) Create a ReplicationEndPoint for meta/look table.
Huaxiang Sun created HBASE-25125: Summary: Create a ReplicationEndPoint for meta/look table. Key: HBASE-25125 URL: https://issues.apache.org/jira/browse/HBASE-25125 Project: HBase Issue Type: Sub-task Components: read replicas Affects Versions: 3.0.0-alpha-1 Reporter: Huaxiang Sun Assignee: Huaxiang Sun -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25126) Add load balance logic in hbase-client to distribute read load over meta replica regions.
Huaxiang Sun created HBASE-25126: Summary: Add load balance logic in hbase-client to distribute read load over meta replica regions. Key: HBASE-25126 URL: https://issues.apache.org/jira/browse/HBASE-25126 Project: HBase Issue Type: Sub-task Affects Versions: 3.0.0-alpha-1 Reporter: Huaxiang Sun Assignee: Huaxiang Sun -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25127) Enhance PerformanceEvaluation to profile meta replica performance.
Huaxiang Sun created HBASE-25127: Summary: Enhance PerformanceEvaluation to profile meta replica performance. Key: HBASE-25127 URL: https://issues.apache.org/jira/browse/HBASE-25127 Project: HBase Issue Type: Sub-task Reporter: Huaxiang Sun Assignee: Huaxiang Sun -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack commented on a change in pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
saintstack commented on a change in pull request #2483:
URL: https://github.com/apache/hbase/pull/2483#discussion_r497723563
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -1,4 +1,4 @@
-/**
+ /**
Review comment:
No need of space here
##
File path:
hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java
##
@@ -571,6 +571,9 @@
String RPC_SCAN_REQUEST_COUNT = "rpcScanRequestCount";
String RPC_SCAN_REQUEST_COUNT_DESC =
"Number of rpc scan requests this RegionServer has answered.";
+ String RPC_FULL_SCAN_REQUEST_COUNT = "rpcFullScanRequestCount";
+ String RPC_FULL_SCAN_REQUEST_COUNT_DESC =
+ "Number of rpc scan requests that were possible full region scans.";
Review comment:
Full Region or full Table?
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -7030,6 +7030,7 @@ public String toString() {
protected Cell joinedContinuationRow = null;
private boolean filterClosed = false;
+protected final byte[] startRow;
Review comment:
Does this need javadoc/comment? It is NOT the same as the
RegionInfo#startRow, right? If so, say so (my first though was why this data
member when we have this info in RegionInfo... but this is the scan startRow...)
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
##
@@ -3498,6 +3501,18 @@ public ScanResponse scan(final RpcController controller,
final ScanRequest reque
throw new ServiceException(e);
}
HRegion region = rsh.r;
+// If the start row is the actual start Row of the region and end row is
actual end row of region
+// or
+// If the stop is not specified or the stop Row is greater than the end
key of the current region
+if ((Bytes.equals(rsh.s.getStartRow(),
region.getRegionInfo().getStartKey())
+ && Bytes.equals(rsh.s.getStopRow(), region.getRegionInfo().getEndKey()))
+ || Bytes.equals(rsh.s.getStopRow(), HConstants.EMPTY_END_ROW)
+ || (Bytes.compareTo(region.getRegionInfo().getEndKey(),
rsh.s.getStopRow()) <= 0
+ && !Bytes.equals(region.getRegionInfo().getEndKey(),
HConstants.EMPTY_END_ROW))) {
+ if(!region.getTableDescriptor().isMetaTable()) {
Review comment:
Why exclude meta table scans? Do this check first before doing all the
above byte array compares?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2482: HBASE-25034:Table regions details on master GUI display slowly.
saintstack commented on a change in pull request #2482:
URL: https://github.com/apache/hbase/pull/2482#discussion_r497728185
##
File path: hbase-server/src/main/resources/hbase-webapps/master/table.jsp
##
@@ -336,12 +323,20 @@
}
%>
- <%= escapeXml(meta.getRegionNameAsString()) %>
- <%=
StringEscapeUtils.escapeHtml4(hostAndPort) %>
- <%= String.format("%,1d", compactingCells)%>
- <%= String.format("%,1d", compactedCells)%>
- <%= String.format("%,1d", compactingCells -
compactedCells)%>
- <%= compactionProgress%>
+ <%= escapeXml(meta.getRegionNameAsString()) %>
+
+
+<%=
StringEscapeUtils.escapeHtml4(
+ hostAndPort) %>
+
+ <%= String.format("%,1d", compactingCells)%>
+
+ <%= String.format("%,1d", compactedCells)%>
+
+ <%= String.format("%,1d", compactingCells - compactedCells)%>
+
+ <%= compactionProgress%>
+
Review comment:
WHy this change in format?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…
Apache-HBase commented on pull request #2451: URL: https://github.com/apache/hbase/pull/2451#issuecomment-701579934 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-18070 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 45s | HBASE-18070 passed | | +1 :green_heart: | checkstyle | 1m 17s | HBASE-18070 passed | | +1 :green_heart: | spotbugs | 2m 13s | HBASE-18070 passed | | -0 :warning: | patch | 2m 22s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 22s | the patch passed | | +1 :green_heart: | checkstyle | 1m 3s | hbase-server: The patch generated 0 new + 42 unchanged - 11 fixed = 42 total (was 53) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 18s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 12s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 34m 11s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2451 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux f68357939a39 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-18070 / 03e8e5b517 | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/5/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25125) Create a ReplicationEndPoint for meta/root table.
[ https://issues.apache.org/jira/browse/HBASE-25125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxiang Sun updated HBASE-25125: - Summary: Create a ReplicationEndPoint for meta/root table. (was: Create a ReplicationEndPoint for meta/look table.) > Create a ReplicationEndPoint for meta/root table. > - > > Key: HBASE-25125 > URL: https://issues.apache.org/jira/browse/HBASE-25125 > Project: HBase > Issue Type: Sub-task > Components: read replicas >Affects Versions: 3.0.0-alpha-1 >Reporter: Huaxiang Sun >Assignee: Huaxiang Sun >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25125) Create a ReplicationEndPoint for meta/root replica replication.
[ https://issues.apache.org/jira/browse/HBASE-25125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxiang Sun updated HBASE-25125: - Summary: Create a ReplicationEndPoint for meta/root replica replication. (was: Create a ReplicationEndPoint for meta/root table.) > Create a ReplicationEndPoint for meta/root replica replication. > --- > > Key: HBASE-25125 > URL: https://issues.apache.org/jira/browse/HBASE-25125 > Project: HBase > Issue Type: Sub-task > Components: read replicas >Affects Versions: 3.0.0-alpha-1 >Reporter: Huaxiang Sun >Assignee: Huaxiang Sun >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack commented on pull request #2481: HBASE-25115 HFilePrettyPrinter can't seek to the row which is the first row of a hfile
saintstack commented on pull request #2481: URL: https://github.com/apache/hbase/pull/2481#issuecomment-701583377 Looks good but why do checks show as failed? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2455: HBASE-25062 The link of "Re:(HBASE-451) Remove HTableDescriptor from …
saintstack commented on pull request #2455: URL: https://github.com/apache/hbase/pull/2455#issuecomment-701586293 Background was that an individual kept making super minor edits to contributors JIRA submissions flooding our mailboxes w/ trivial changes making it hard to find substance. The comment is about discouraging the behavior. The discussion pointed to I don't recall. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
