[GitHub] [hbase] Apache-HBase commented on pull request #2582: HBASE-25187 Improve SizeCachedKV variants initialization
Apache-HBase commented on pull request #2582: URL: https://github.com/apache/hbase/pull/2582#issuecomment-715010562 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 44s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 3s | master passed | | +1 :green_heart: | checkstyle | 1m 52s | master passed | | +1 :green_heart: | spotbugs | 3m 24s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 40s | the patch passed | | -0 :warning: | checkstyle | 0m 27s | hbase-common: The patch generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) | | -0 :warning: | checkstyle | 1m 20s | hbase-server: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 24m 53s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | -1 :x: | spotbugs | 1m 13s | hbase-common generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 25s | The patch does not generate ASF License warnings. | | | | 59m 14s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-common | | | org.apache.hadoop.hbase.SizeCachedByteBufferKeyValue doesn't override ByteBufferKeyValue.equals(Object) At SizeCachedByteBufferKeyValue.java:At SizeCachedByteBufferKeyValue.java:[line 1] | | | Dead store to keyLen rather than field with same name in new org.apache.hadoop.hbase.SizeCachedKeyValue(byte[], int, int, long, int) At SizeCachedKeyValue.java:than field with same name in new org.apache.hadoop.hbase.SizeCachedKeyValue(byte[], int, int, long, int) At SizeCachedKeyValue.java:[line 47] | | | org.apache.hadoop.hbase.SizeCachedNoTagsByteBufferKeyValue doesn't override ByteBufferKeyValue.equals(Object) At SizeCachedNoTagsByteBufferKeyValue.java:At SizeCachedNoTagsByteBufferKeyValue.java:[line 1] | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2582 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 9341357e26a4 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / d8de24c311 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-common.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/1/artifact/yetus-general-check/output/new-spotbugs-hbase-common.html | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2582: HBASE-25187 Improve SizeCachedKV variants initialization
Apache-HBase commented on pull request #2582: URL: https://github.com/apache/hbase/pull/2582#issuecomment-714996647 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 53s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 39s | master passed | | +1 :green_heart: | compile | 1m 34s | master passed | | +1 :green_heart: | shadedjars | 7m 59s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 4s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 29s | the patch passed | | +1 :green_heart: | compile | 1m 19s | the patch passed | | +1 :green_heart: | javac | 1m 19s | the patch passed | | +1 :green_heart: | shadedjars | 6m 28s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 24s | hbase-common in the patch passed. | | -1 :x: | unit | 18m 13s | hbase-server in the patch failed. | | | | 51m 18s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2582 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f1e0b5cf7052 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / d8de24c311 | | Default Java | 1.8.0_232 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/1/testReport/ | | Max. process+thread count | 978 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2582: HBASE-25187 Improve SizeCachedKV variants initialization
Apache-HBase commented on pull request #2582: URL: https://github.com/apache/hbase/pull/2582#issuecomment-714998157 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 16s | master passed | | +1 :green_heart: | compile | 1m 30s | master passed | | +1 :green_heart: | shadedjars | 6m 50s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 6s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 4s | the patch passed | | +1 :green_heart: | compile | 1m 31s | the patch passed | | +1 :green_heart: | javac | 1m 31s | the patch passed | | +1 :green_heart: | shadedjars | 6m 40s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 43s | hbase-common in the patch passed. | | -1 :x: | unit | 20m 4s | hbase-server in the patch failed. | | | | 52m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2582 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1d9440f66db0 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / d8de24c311 | | Default Java | 2020-01-14 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/1/testReport/ | | Max. process+thread count | 1004 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25192) s390x support for HBase CI
[ https://issues.apache.org/jira/browse/HBASE-25192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219493#comment-17219493 ] Aditi Jadhav commented on HBASE-25192: -- Yes. It will be good to have nightly builds running on s390x. We could provide a s390x vm for adding a slave node. Regarding platfrom specific issues i think the jenkins build logs should give a good enough insight. I am also wondering how developers deal with issue on arm? > s390x support for HBase CI > -- > > Key: HBASE-25192 > URL: https://issues.apache.org/jira/browse/HBASE-25192 > Project: HBase > Issue Type: Improvement >Reporter: Aditi Jadhav >Priority: Major > > Hi All, > As I understand that currently CI available for HBase ( to validate any code > updates ) is : https://ci-hadoop.apache.org/job/HBase/ and as can be seen > most of the deployment here is on amd64 and ppc64le. > Wanted to know your views/comments/suggestions for extending CI of HBase on > s390x . > 1) If the community will be willing to add - vm's/slaves ( s390x based ) to > their current above CI. > 2) Use an externally hosted jenkins CI - for running HBase build on s390x and > link the results of the build to the above CI. > Please let me know your thoughts. > Regards, > Aditi -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2569: HBASE-25206 Data loss can happen if a cloned table loses original spl…
Apache-HBase commented on pull request #2569: URL: https://github.com/apache/hbase/pull/2569#issuecomment-714939895 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 24s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | shadedjars | 6m 51s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 41s | the patch passed | | +1 :green_heart: | compile | 1m 0s | the patch passed | | +1 :green_heart: | javac | 1m 0s | the patch passed | | +1 :green_heart: | shadedjars | 6m 57s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 146m 52s | hbase-server in the patch failed. | | | | 172m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2569 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 293b21117b6f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / d8de24c311 | | Default Java | 1.8.0_232 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/testReport/ | | Max. process+thread count | 4751 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2569: HBASE-25206 Data loss can happen if a cloned table loses original spl…
Apache-HBase commented on pull request #2569: URL: https://github.com/apache/hbase/pull/2569#issuecomment-714939554 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 57s | master passed | | +1 :green_heart: | compile | 1m 7s | master passed | | +1 :green_heart: | shadedjars | 6m 38s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 58s | the patch passed | | +1 :green_heart: | compile | 1m 5s | the patch passed | | +1 :green_heart: | javac | 1m 5s | the patch passed | | +1 :green_heart: | shadedjars | 6m 35s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 143m 45s | hbase-server in the patch failed. | | | | 171m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2569 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7e3a58a3529c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / d8de24c311 | | Default Java | 2020-01-14 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/testReport/ | | Max. process+thread count | 3959 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ramkrish86 opened a new pull request #2582: Hbase 25187 master
ramkrish86 opened a new pull request #2582: URL: https://github.com/apache/hbase/pull/2582 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25187) Improve SizeCachedKV variants initialization
[ https://issues.apache.org/jira/browse/HBASE-25187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219459#comment-17219459 ] ramkrishna.s.vasudevan commented on HBASE-25187: This patch along with the removal of the sanity check in the KVUtil gives 12% improvement in scans reading 5G of data and filtering using SCVF. How ever in this patch I have not removed the sanity check so probably the improvement is going to be lesser than 12%. > Improve SizeCachedKV variants initialization > > > Key: HBASE-25187 > URL: https://issues.apache.org/jira/browse/HBASE-25187 > Project: HBase > Issue Type: Improvement >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.3.3 > > > Currently in SizeCachedKV we get the rowlength and Key length from the > buffers. This can be optimized because we can pass the keylen and row len > while actually creating the cell while reading the cell from the block. Some > times we see that the SizeCachedKV takes the max width in a flame graph - > considering the fact we also do a sanity check on the created KV. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25206) Data loss can happen if a cloned table loses original split region(delete table)
[
https://issues.apache.org/jira/browse/HBASE-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219457#comment-17219457
]
Anoop Sam John commented on HBASE-25206:
Tks. [~brfrn169]
So even if snapshot cloned to new table or not, this case will cause data loss
from snapshot.
> Data loss can happen if a cloned table loses original split region(delete
> table)
>
>
> Key: HBASE-25206
> URL: https://issues.apache.org/jira/browse/HBASE-25206
> Project: HBase
> Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
>
> Steps to reproduce are as follows:
> 1. Create a table and put some data into the table:
> {code:java}
> create 'test1','cf'
> put 'test1','r1','cf','v1'
> put 'test1','r2','cf','v2'
> put 'test1','r3','cf','v3'
> put 'test1','r4','cf','v4'
> put 'test1','r5','cf','v5'
> {code}
> 2. Take a snapshot for the table:
> {code:java}
> snapshot 'test1','snap_test'
> {code}
> 3. Clone the snapshot to another table
> {code:java}
> clone_snapshot 'snap_test','test2'
> {code}
> 4. Split the original table
> {code:java}
> split 'test1','r3'
> {code}
> 5. Drop the original table
> {code:java}
> disable 'test1'
> drop 'test1'
> {code}
> After that, we see the error like the following in RS log when opening the
> regions of the cloned table:
> {code:java}
> 2020-10-20 13:32:18,415 WARN org.apache.hadoop.hbase.regionserver.HRegion:
> Failed initialize of region=
> test2,,1603200595702.bebdc4f740626206eeccad96b7643261., starting to roll back
> memstore
> java.io.IOException: java.io.IOException: java.io.FileNotFoundException:
> Unable to open link: org.apache.hadoop.hbase.io.HFileLink
> locations=[hdfs:// HOST>:8020/hbase/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/.tmp/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/mobdir/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/archive/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89]
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeStores(HRegion.java:1095)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:943)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:899)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7246)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7204)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7176)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7134)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7085)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:283)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:108)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.io.FileNotFoundException: Unable to open
> link: org.apache.hadoop.hbase.io.HFileLink locations=[hdfs:// HOST>:8020/hbase/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/.tmp/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/mobdir/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/archive/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89]
> at
> org.apache.hadoop.hbase.regionserver.HStore.openStoreFiles(HStore.java:590)
> at
> org.apache.hadoop.hbase.regionserver.HStore.loadStoreFiles(HStore.java:557)
> at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:303)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:5731)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1059)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1056)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$
[GitHub] [hbase] Apache-HBase commented on pull request #2569: HBASE-25206 Data loss can happen if a cloned table loses original spl…
Apache-HBase commented on pull request #2569: URL: https://github.com/apache/hbase/pull/2569#issuecomment-714895398 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 24s | master passed | | +1 :green_heart: | checkstyle | 1m 2s | master passed | | +1 :green_heart: | spotbugs | 2m 1s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 22s | the patch passed | | +1 :green_heart: | checkstyle | 1m 3s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 16m 55s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 38m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2569 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 354b4c290bbd 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / d8de24c311 | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2575: HBASE-25212 [branch-2] Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2575: URL: https://github.com/apache/hbase/pull/2575#issuecomment-714894728 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 30s | branch-2 passed | | +1 :green_heart: | compile | 1m 9s | branch-2 passed | | +1 :green_heart: | shadedjars | 7m 23s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 28s | the patch passed | | +1 :green_heart: | compile | 1m 10s | the patch passed | | +1 :green_heart: | javac | 1m 10s | the patch passed | | +1 :green_heart: | shadedjars | 7m 25s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 193m 36s | hbase-server in the patch passed. | | | | 224m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2575 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux cc1edd6b6e36 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / fdea847130 | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/3/testReport/ | | Max. process+thread count | 3021 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-25206) Data loss can happen if a cloned table loses original split region(delete table)
[
https://issues.apache.org/jira/browse/HBASE-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219440#comment-17219440
]
Toshihiro Suzuki edited comment on HBASE-25206 at 10/23/20, 3:38 AM:
-
[~anoop.hbase] Sorry. I just noticed your comment.
This issue happens because when deleting a table, DeleteTableProcedure archives
only online regions of the table (that don't include OFFLINE or SPLIT regions).
The timeline of this issue is as follows:
1. Create a table that has RegionA
{code:java}
Original table:
RegionA
{code}
2. Take a snapshot for the table
{code:java}
Original table:
RegionA
Snapshot:
Reference to RegionA
{code}
3. Clone the snapshot to another table
{code:java}
Original table:
RegionA
Cloned table:
Reference to RegionA
Snapshot:
Reference to RegionA
{code}
4. Split the original table. The state of RegionA becomes SPLIT, and RegionB
and RegionC are the daughter regions of RegionA:
{code:java}
Original table:
RegionA <- SPLIT state
RegionB
RegionC
Cloned table:
Reference to RegionA
Snapshot:
Reference to RegionA
{code}
5. Drop the original table. However, only RegionB and RegionC are archived due
to the bug. As a result, the cloned table gets data lost. Also the snapshot
gets corrupted:
{code:java}
Cloned table:
Reference to RegionA
Snapshot:
Reference to RegionA
Archive:
RegionB
RegionC
{code}
was (Author: brfrn169):
[~anoop.hbase] Sorry. I just noticed your comment.
This issue happens because when deleting a table, DeleteTableProcedure archives
only online regions of the table (that don't include OFFLINE or SPLITTING
regions).
The timeline of this issue is as follows:
1. Create a table that has RegionA
{code:java}
Original table:
RegionA
{code}
2. Take a snapshot for the table
{code:java}
Original table:
RegionA
Snapshot:
Reference to RegionA
{code}
3. Clone the snapshot to another table
{code:java}
Original table:
RegionA
Cloned table:
Reference to RegionA
Snapshot:
Reference to RegionA
{code}
4. Split the original table. The state of RegionA becomes SPLIT, and RegionB
and RegionC are the daughter regions of RegionA:
{code:java}
Original table:
RegionA <- SPLIT state
RegionB
RegionC
Cloned table:
Reference to RegionA
Snapshot:
Reference to RegionA
{code}
5. Drop the original table. However, only RegionB and RegionC are archived due
to the bug. As a result, the cloned table gets data lost. Also the snapshot
gets corrupted:
{code:java}
Cloned table:
Reference to RegionA
Snapshot:
Reference to RegionA
Archive:
RegionB
RegionC
{code}
> Data loss can happen if a cloned table loses original split region(delete
> table)
>
>
> Key: HBASE-25206
> URL: https://issues.apache.org/jira/browse/HBASE-25206
> Project: HBase
> Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
>
> Steps to reproduce are as follows:
> 1. Create a table and put some data into the table:
> {code:java}
> create 'test1','cf'
> put 'test1','r1','cf','v1'
> put 'test1','r2','cf','v2'
> put 'test1','r3','cf','v3'
> put 'test1','r4','cf','v4'
> put 'test1','r5','cf','v5'
> {code}
> 2. Take a snapshot for the table:
> {code:java}
> snapshot 'test1','snap_test'
> {code}
> 3. Clone the snapshot to another table
> {code:java}
> clone_snapshot 'snap_test','test2'
> {code}
> 4. Split the original table
> {code:java}
> split 'test1','r3'
> {code}
> 5. Drop the original table
> {code:java}
> disable 'test1'
> drop 'test1'
> {code}
> After that, we see the error like the following in RS log when opening the
> regions of the cloned table:
> {code:java}
> 2020-10-20 13:32:18,415 WARN org.apache.hadoop.hbase.regionserver.HRegion:
> Failed initialize of region=
> test2,,1603200595702.bebdc4f740626206eeccad96b7643261., starting to roll back
> memstore
> java.io.IOException: java.io.IOException: java.io.FileNotFoundException:
> Unable to open link: org.apache.hadoop.hbase.io.HFileLink
> locations=[hdfs:// HOST>:8020/hbase/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/.tmp/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/mobdir/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/archive/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89]
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeStores(HRegion.java:1095)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:943)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:899)
> at
> org.apache.
[jira] [Commented] (HBASE-25206) Data loss can happen if a cloned table loses original split region(delete table)
[
https://issues.apache.org/jira/browse/HBASE-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219440#comment-17219440
]
Toshihiro Suzuki commented on HBASE-25206:
--
[~anoop.hbase] Sorry. I just noticed your comment.
This issue happens because when deleting a table, DeleteTableProcedure archives
only online regions of the table (that don't include OFFLINE or SPLITTING
regions).
The timeline of this issue is as follows:
1. Create a table that has RegionA
{code:java}
Original table:
RegionA
{code}
2. Take a snapshot for the table
{code:java}
Original table:
RegionA
Snapshot:
Reference to RegionA
{code}
3. Clone the snapshot to another table
{code:java}
Original table:
RegionA
Cloned table:
Reference to RegionA
Snapshot:
Reference to RegionA
{code}
4. Split the original table. The state of RegionA becomes SPLIT, and RegionB
and RegionC are the daughter regions of RegionA:
{code:java}
Original table:
RegionA <- SPLIT state
RegionB
RegionC
Cloned table:
Reference to RegionA
Snapshot:
Reference to RegionA
{code}
5. Drop the original table. However, only RegionB and RegionC are archived due
to the bug. As a result, the cloned table gets data lost. Also the snapshot
gets corrupted:
{code:java}
Cloned table:
Reference to RegionA
Snapshot:
Reference to RegionA
Archive:
RegionB
RegionC
{code}
> Data loss can happen if a cloned table loses original split region(delete
> table)
>
>
> Key: HBASE-25206
> URL: https://issues.apache.org/jira/browse/HBASE-25206
> Project: HBase
> Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
>
> Steps to reproduce are as follows:
> 1. Create a table and put some data into the table:
> {code:java}
> create 'test1','cf'
> put 'test1','r1','cf','v1'
> put 'test1','r2','cf','v2'
> put 'test1','r3','cf','v3'
> put 'test1','r4','cf','v4'
> put 'test1','r5','cf','v5'
> {code}
> 2. Take a snapshot for the table:
> {code:java}
> snapshot 'test1','snap_test'
> {code}
> 3. Clone the snapshot to another table
> {code:java}
> clone_snapshot 'snap_test','test2'
> {code}
> 4. Split the original table
> {code:java}
> split 'test1','r3'
> {code}
> 5. Drop the original table
> {code:java}
> disable 'test1'
> drop 'test1'
> {code}
> After that, we see the error like the following in RS log when opening the
> regions of the cloned table:
> {code:java}
> 2020-10-20 13:32:18,415 WARN org.apache.hadoop.hbase.regionserver.HRegion:
> Failed initialize of region=
> test2,,1603200595702.bebdc4f740626206eeccad96b7643261., starting to roll back
> memstore
> java.io.IOException: java.io.IOException: java.io.FileNotFoundException:
> Unable to open link: org.apache.hadoop.hbase.io.HFileLink
> locations=[hdfs:// HOST>:8020/hbase/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/.tmp/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/mobdir/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/archive/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89]
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeStores(HRegion.java:1095)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:943)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:899)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7246)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7204)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7176)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7134)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7085)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:283)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:108)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.io.FileNotFoundException: Unable to open
> link: org.apache.hadoop.hbase.io.HFileLink locations=[hdfs:// HOST>:8020/hbase/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710
[GitHub] [hbase] Apache-HBase commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-714886902 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 32s | master passed | | +1 :green_heart: | compile | 1m 12s | master passed | | +1 :green_heart: | shadedjars | 7m 22s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 36s | the patch passed | | +1 :green_heart: | compile | 1m 12s | the patch passed | | +1 :green_heart: | javac | 1m 12s | the patch passed | | +1 :green_heart: | shadedjars | 7m 18s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 47s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 202m 49s | hbase-server in the patch passed. | | | | 233m 29s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2574 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux de6dec5f7388 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/3/testReport/ | | Max. process+thread count | 3670 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25206) Data loss can happen if a cloned table loses original split region(delete table)
[
https://issues.apache.org/jira/browse/HBASE-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki updated HBASE-25206:
-
Description:
Steps to reproduce are as follows:
1. Create a table and put some data into the table:
{code:java}
create 'test1','cf'
put 'test1','r1','cf','v1'
put 'test1','r2','cf','v2'
put 'test1','r3','cf','v3'
put 'test1','r4','cf','v4'
put 'test1','r5','cf','v5'
{code}
2. Take a snapshot for the table:
{code:java}
snapshot 'test1','snap_test'
{code}
3. Clone the snapshot to another table
{code:java}
clone_snapshot 'snap_test','test2'
{code}
4. Split the original table
{code:java}
split 'test1','r3'
{code}
5. Drop the original table
{code:java}
disable 'test1'
drop 'test1'
{code}
After that, we see the error like the following in RS log when opening the
regions of the cloned table:
{code:java}
2020-10-20 13:32:18,415 WARN org.apache.hadoop.hbase.regionserver.HRegion:
Failed initialize of region=
test2,,1603200595702.bebdc4f740626206eeccad96b7643261., starting to roll back
memstore
java.io.IOException: java.io.IOException: java.io.FileNotFoundException: Unable
to open link: org.apache.hadoop.hbase.io.HFileLink locations=[hdfs://:8020/hbase/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
hdfs://:8020/hbase/.tmp/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
hdfs://:8020/hbase/mobdir/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
hdfs://:8020/hbase/archive/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89]
at
org.apache.hadoop.hbase.regionserver.HRegion.initializeStores(HRegion.java:1095)
at
org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:943)
at
org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:899)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7246)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7204)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7176)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7134)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7085)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:283)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:108)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: java.io.FileNotFoundException: Unable to open
link: org.apache.hadoop.hbase.io.HFileLink locations=[hdfs://:8020/hbase/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
hdfs://:8020/hbase/.tmp/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
hdfs://:8020/hbase/mobdir/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
hdfs://:8020/hbase/archive/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89]
at
org.apache.hadoop.hbase.regionserver.HStore.openStoreFiles(HStore.java:590)
at
org.apache.hadoop.hbase.regionserver.HStore.loadStoreFiles(HStore.java:557)
at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:303)
at
org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:5731)
at
org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1059)
at
org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1056)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Caused by: java.io.FileNotFoundException: Unable to open link:
org.apache.hadoop.hbase.io.HFileLink locations=[hdfs://:8020/hbase/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
hdfs://:8020/hbase/.tmp/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
hdfs://:8020/hbase/mobdir/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
hdfs://:8020/hbase/archive/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89]
at
org.apache.hadoop.hbase.io.FileLink$FileLinkInputStream.tryOpen(FileLink.java:322)
at
org.apache.hado
[GitHub] [hbase] Apache-HBase commented on pull request #2575: HBASE-25212 [branch-2] Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2575: URL: https://github.com/apache/hbase/pull/2575#issuecomment-714878901 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 40s | branch-2 passed | | +1 :green_heart: | compile | 0m 56s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 57s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 20s | the patch passed | | +1 :green_heart: | compile | 0m 56s | the patch passed | | +1 :green_heart: | javac | 0m 56s | the patch passed | | +1 :green_heart: | shadedjars | 6m 1s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 144m 2s | hbase-server in the patch passed. | | | | 169m 3s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2575 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 0fb47c446340 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / fdea847130 | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/3/testReport/ | | Max. process+thread count | 4156 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-25206) Data loss can happen if a cloned table loses original split region(delete table)
[
https://issues.apache.org/jira/browse/HBASE-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17218830#comment-17218830
]
Anoop Sam John edited comment on HBASE-25206 at 10/23/20, 3:01 AM:
---
the data loss (file deleted instead of archived) will happen if we took
snapshot and keep that only but deleted the original table right? Did not clone
new table from snapshot initially but later at some point user may want to do
so. Just confirming.
[~brfrn169]
was (Author: anoop.hbase):
the data loss (file deleted instead of archived) will happen if we took
snapshot and keep that only but deleted the original table right? Did not clone
new table from snapshot initially but later at some point user may want to do
so. Just confirming.
> Data loss can happen if a cloned table loses original split region(delete
> table)
>
>
> Key: HBASE-25206
> URL: https://issues.apache.org/jira/browse/HBASE-25206
> Project: HBase
> Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
>
> Steps to reproduce are as follows:
> 1. Create a table and put some data into the table:
> {code:java}
> create 'test1','cf'
> put 'test1','r1','cf','v1'
> put 'test1','r2','cf','v2'
> put 'test1','r3','cf','v3'
> put 'test1','r4','cf','v4'
> put 'test1','r5','cf','v5'
> {code}
> 2. Take a snapshot for the table:
> {code:java}
> snapshot 'test1','snap_test'
> {code}
> 3. Clone the snapshot to another table
> {code:java}
> clone_snapshot 'snap_test','test2'
> {code}
> 4. Delete the snapshot
> {code:java}
> delete_snapshot 'snap_test'
> {code}
> 5. Split the original table
> {code:java}
> split 'test1','r3'
> {code}
> 6. Drop the original table
> {code:java}
> disable 'test1'
> drop 'test1'
> {code}
> After that, we see the error like the following in RS log when opening the
> regions of the cloned table:
> {code:java}
> 2020-10-20 13:32:18,415 WARN org.apache.hadoop.hbase.regionserver.HRegion:
> Failed initialize of region=
> test2,,1603200595702.bebdc4f740626206eeccad96b7643261., starting to roll back
> memstore
> java.io.IOException: java.io.IOException: java.io.FileNotFoundException:
> Unable to open link: org.apache.hadoop.hbase.io.HFileLink
> locations=[hdfs:// HOST>:8020/hbase/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/.tmp/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/mobdir/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/archive/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89]
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeStores(HRegion.java:1095)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:943)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:899)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7246)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7204)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7176)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7134)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7085)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:283)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:108)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.io.FileNotFoundException: Unable to open
> link: org.apache.hadoop.hbase.io.HFileLink locations=[hdfs:// HOST>:8020/hbase/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/.tmp/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/mobdir/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89,
> hdfs:// HOST>:8020/hbase/archive/data/default/test1/349b766b1b38e21f627ed4e441ae643c/cf/b6e39865710345c8998dec0bcc94cc89]
> at
> org.apache.hadoop.hbase.regionserver.HStore.openStoreFiles(HStore.
[GitHub] [hbase] Apache-HBase commented on pull request #2576: HBASE-25212 [branch-1] Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2576: URL: https://github.com/apache/hbase/pull/2576#issuecomment-714877013 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 5s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 7 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 9m 40s | branch-1 passed | | +1 :green_heart: | compile | 0m 42s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | compile | 0m 48s | branch-1 passed with JDK v1.7.0_272 | | +1 :green_heart: | checkstyle | 2m 5s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 19s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 47s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 42s | branch-1 passed with JDK v1.7.0_272 | | +0 :ok: | spotbugs | 3m 16s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 12s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 3s | the patch passed | | +1 :green_heart: | compile | 0m 42s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javac | 0m 42s | the patch passed | | +1 :green_heart: | compile | 0m 47s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | javac | 0m 47s | the patch passed | | -1 :x: | checkstyle | 1m 54s | hbase-server: The patch generated 7 new + 820 unchanged - 10 fixed = 827 total (was 830) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 53s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 32s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 42s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | findbugs | 3m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 111m 30s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 29s | The patch does not generate ASF License warnings. | | | | 155m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2576/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2576 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 973c811e006d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2576/out/precommit/personality/provided.sh | | git revision | branch-1 / 8e387e9 | | Default Java | 1.7.0_272 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2576/3/artifact/out/diff-checkstyle-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2576/3/testReport/ | | Max. process+thread count | 3542 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2576/3/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] brfrn169 commented on pull request #2569: HBASE-25206 Data loss can happen if a cloned table loses original spl…
brfrn169 commented on pull request #2569: URL: https://github.com/apache/hbase/pull/2569#issuecomment-714874912 @Apache9 I just modified the patch for your review. Can you please review it when you get a chance? Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-714867315 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 48s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | shadedjars | 6m 39s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 30s | the patch passed | | +1 :green_heart: | compile | 0m 56s | the patch passed | | +1 :green_heart: | javac | 0m 56s | the patch passed | | +1 :green_heart: | shadedjars | 6m 31s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 140m 52s | hbase-server in the patch failed. | | | | 167m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2574 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a0883d1d9fab 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | Default Java | 1.8.0_232 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/3/testReport/ | | Max. process+thread count | 4402 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
Apache-HBase commented on pull request #2483: URL: https://github.com/apache/hbase/pull/2483#issuecomment-714860665 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 25s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 50s | master passed | | +1 :green_heart: | compile | 1m 16s | master passed | | +1 :green_heart: | shadedjars | 7m 7s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 53s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 49s | the patch passed | | +1 :green_heart: | compile | 1m 15s | the patch passed | | +1 :green_heart: | javac | 1m 15s | the patch passed | | +1 :green_heart: | shadedjars | 7m 5s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 36s | hbase-hadoop-compat in the patch passed. | | -1 :x: | unit | 211m 28s | hbase-server in the patch failed. | | | | 243m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/16/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2483 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 002ed7094015 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | Default Java | 1.8.0_232 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/16/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/16/testReport/ | | Max. process+thread count | 3637 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/16/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #2556: HBASE-25193: Add support for row prefix and type in the WAL Pretty Printer
Apache9 commented on a change in pull request #2556:
URL: https://github.com/apache/hbase/pull/2556#discussion_r510549322
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALPrettyPrinter.java
##
@@ -301,15 +318,12 @@ public void processFile(final Configuration conf, final
Path p)
List> actions = new ArrayList<>();
for (Cell cell : edit.getCells()) {
// add atomic operation to txn
- Map op = new HashMap<>(toStringMap(cell,
outputOnlyRowKey));
- if (outputValues) {
-op.put("value", Bytes.toStringBinary(CellUtil.cloneValue(cell)));
- }
- // check row output filter
- if (row == null || ((String) op.get("row")).equals(row)) {
Review comment:
OK, got it. No problem.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2542: HBASE-24667 Rename configs that support atypical DNS set ups to put them in hbase.unsafe
Apache-HBase commented on pull request #2542: URL: https://github.com/apache/hbase/pull/2542#issuecomment-714852038 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 43s | master passed | | +1 :green_heart: | compile | 1m 43s | master passed | | +1 :green_heart: | shadedjars | 6m 39s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 20s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 32s | the patch passed | | +1 :green_heart: | compile | 1m 45s | the patch passed | | +1 :green_heart: | javac | 1m 45s | the patch passed | | +1 :green_heart: | shadedjars | 6m 32s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 24s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 142m 38s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 4m 4s | hbase-rest in the patch passed. | | | | 178m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2542 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7e3da2543fa3 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/3/testReport/ | | Max. process+thread count | 4819 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server hbase-rest U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25207) Revisit the implementation and usage of RegionStates.include
[
https://issues.apache.org/jira/browse/HBASE-25207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219400#comment-17219400
]
Hudson commented on HBASE-25207:
Results for branch branch-2.2
[build #105 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/105/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/105//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/105//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/105//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Revisit the implementation and usage of RegionStates.include
>
>
> Key: HBASE-25207
> URL: https://issues.apache.org/jira/browse/HBASE-25207
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7
>
>
> After several round of refactoring and fixing, the method has been used in
> lots of places and the implementation looks really confusing.
> As in the first if condition for testing RegionStateNode and RegionInfo
> state, we will always return false when split is true, which means we will
> always filter out split parent, as a split parent, is split = true and also
> offline = true.
> I think the reason why there is no problem is that, only in
> EnableTableProcedure we call this method with offline = true, and
> EnableTableProcedure does not need to deal with split parent...
> And now since we found a problem in HBASE-25206, where we need to get split
> parent when deleting a table, I think it is time to revisit this method and
> make logic less confusing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2542: HBASE-24667 Rename configs that support atypical DNS set ups to put them in hbase.unsafe
Apache-HBase commented on pull request #2542: URL: https://github.com/apache/hbase/pull/2542#issuecomment-714849144 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 7s | master passed | | +1 :green_heart: | compile | 1m 55s | master passed | | +1 :green_heart: | shadedjars | 6m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 26s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 0s | the patch passed | | +1 :green_heart: | compile | 1m 58s | the patch passed | | +1 :green_heart: | javac | 1m 58s | the patch passed | | +1 :green_heart: | shadedjars | 6m 34s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 45s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 131m 26s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 2m 58s | hbase-rest in the patch passed. | | | | 167m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2542 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5ca81c7d146a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/3/testReport/ | | Max. process+thread count | 3918 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server hbase-rest U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-25110) Add heartbeat for ReplicationServer and dispatch replication sources to ReplicationServer
[ https://issues.apache.org/jira/browse/HBASE-25110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-25110: -- Assignee: Guanghao Zhang > Add heartbeat for ReplicationServer and dispatch replication sources to > ReplicationServer > - > > Key: HBASE-25110 > URL: https://issues.apache.org/jira/browse/HBASE-25110 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2575: HBASE-25212 [branch-2] Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2575: URL: https://github.com/apache/hbase/pull/2575#issuecomment-714841827 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 52s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 42s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 35s | branch-2 passed | | +1 :green_heart: | spotbugs | 2m 42s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 25s | the patch passed | | -0 :warning: | checkstyle | 1m 42s | hbase-server: The patch generated 3 new + 479 unchanged - 2 fixed = 482 total (was 481) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 30s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 39m 25s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2575 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 219dee05dbae 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / fdea847130 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-25128) RSGroupInfo's toString() and hashCode() does not take into account configuration map.
[ https://issues.apache.org/jira/browse/HBASE-25128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25128. Fix Version/s: 2.4.0 3.0.0-alpha-1 Resolution: Fixed Pushed to branch-2 and master. Thanks [~sanjeetnishad] for contributing. > RSGroupInfo's toString() and hashCode() does not take into account > configuration map. > - > > Key: HBASE-25128 > URL: https://issues.apache.org/jira/browse/HBASE-25128 > Project: HBase > Issue Type: Improvement > Components: rsgroup >Affects Versions: 2.2.3 >Reporter: Sanjeet Nishad >Assignee: Sanjeet Nishad >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.4.0 > > > RSGroupInfo's toString() and hashcode() methods should include Configurations > as well which is added as a part of HBASE-24431. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
Apache-HBase commented on pull request #2483: URL: https://github.com/apache/hbase/pull/2483#issuecomment-714838976 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 57s | master passed | | +1 :green_heart: | compile | 1m 25s | master passed | | +1 :green_heart: | shadedjars | 6m 38s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 2s | the patch passed | | +1 :green_heart: | compile | 1m 25s | the patch passed | | +1 :green_heart: | javac | 1m 25s | the patch passed | | +1 :green_heart: | shadedjars | 6m 37s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 0s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 35s | hbase-hadoop-compat in the patch passed. | | -1 :x: | unit | 133m 42s | hbase-server in the patch failed. | | | | 163m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/16/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2483 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 096ad71cb6f0 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | Default Java | 2020-01-14 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/16/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/16/testReport/ | | Max. process+thread count | 4680 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/16/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] infraio merged pull request #2484: HBASE-25128 RSGroupInfo's toString() and hashCode() does not take int…
infraio merged pull request #2484: URL: https://github.com/apache/hbase/pull/2484 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-714834407 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 46s | master passed | | +1 :green_heart: | checkstyle | 1m 18s | master passed | | +1 :green_heart: | spotbugs | 2m 4s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 49s | the patch passed | | +1 :green_heart: | checkstyle | 1m 16s | hbase-server: The patch generated 0 new + 409 unchanged - 2 fixed = 409 total (was 411) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 45s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 16s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. | | | | 47m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2574 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux b5dfd053fcca 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2576: HBASE-25212 [branch-1] Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2576: URL: https://github.com/apache/hbase/pull/2576#issuecomment-714833531 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 11m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 5 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 9m 54s | branch-1 passed | | +1 :green_heart: | compile | 0m 41s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | compile | 0m 47s | branch-1 passed with JDK v1.7.0_272 | | +1 :green_heart: | checkstyle | 2m 3s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 17s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 43s | branch-1 passed with JDK v1.7.0_272 | | +0 :ok: | spotbugs | 3m 14s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 10s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 5s | the patch passed | | +1 :green_heart: | compile | 0m 41s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javac | 0m 41s | the patch passed | | +1 :green_heart: | compile | 0m 48s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | javac | 0m 48s | the patch passed | | -1 :x: | checkstyle | 1m 55s | hbase-server: The patch generated 9 new + 797 unchanged - 10 fixed = 806 total (was 807) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 5m 0s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 31s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 43s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | findbugs | 3m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 111m 20s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 29s | The patch does not generate ASF License warnings. | | | | 166m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2576/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2576 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 48a15f9d25e6 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2576/out/precommit/personality/provided.sh | | git revision | branch-1 / 8e387e9 | | Default Java | 1.7.0_272 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2576/2/artifact/out/diff-checkstyle-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2576/2/testReport/ | | Max. process+thread count | 3542 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2576/2/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2575: HBASE-25212 [branch-2] Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2575: URL: https://github.com/apache/hbase/pull/2575#issuecomment-714827709 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 9s | branch-2 passed | | +1 :green_heart: | compile | 1m 0s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 53s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 47s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 36s | the patch passed | | +1 :green_heart: | compile | 1m 18s | the patch passed | | +1 :green_heart: | javac | 1m 18s | the patch passed | | +1 :green_heart: | shadedjars | 7m 41s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 45s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 230m 7s | hbase-server in the patch passed. | | | | 260m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2575 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux fa90dd9206d5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / fdea847130 | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/2/testReport/ | | Max. process+thread count | 2481 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sandeepvinayak commented on a change in pull request #2556: HBASE-25193: Add support for row prefix and type in the WAL Pretty Printer
sandeepvinayak commented on a change in pull request #2556:
URL: https://github.com/apache/hbase/pull/2556#discussion_r510491746
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALPrettyPrinter.java
##
@@ -301,15 +318,12 @@ public void processFile(final Configuration conf, final
Path p)
List> actions = new ArrayList<>();
for (Cell cell : edit.getCells()) {
// add atomic operation to txn
- Map op = new HashMap<>(toStringMap(cell,
outputOnlyRowKey));
- if (outputValues) {
-op.put("value", Bytes.toStringBinary(CellUtil.cloneValue(cell)));
- }
- // check row output filter
- if (row == null || ((String) op.get("row")).equals(row)) {
Review comment:
I see what you mean, I changed the flow because it was not very readable
even though it was functionally correct. Here is why I thought so:
older version
```java
Map op = new HashMap<>(toStringMap(cell,
outputOnlyRowKey));
if (outputValues) {
op.put("value", Bytes.toStringBinary(CellUtil.cloneValue(cell)));
}
// check row output filter
if (row == null || ((String) op.get("row")).equals(row)) {
actions.add(op);
}
op.put("total_size_sum", cell.heapSize());
```
Here, we are computing the stringMap and then checking if we need this row,
and then even if don't need the row we still compute `total_size_sum`. Not sure
why it was done this way but this was my concern. Hope it answers your question.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
Apache-HBase commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-714822872 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 26s | master passed | | +1 :green_heart: | compile | 2m 19s | master passed | | +1 :green_heart: | shadedjars | 6m 33s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 17s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 20s | the patch passed | | +1 :green_heart: | compile | 2m 18s | the patch passed | | +1 :green_heart: | javac | 2m 18s | the patch passed | | +1 :green_heart: | shadedjars | 6m 28s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 14s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 295m 19s | root in the patch failed. | | | | 327m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/14/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2308 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 2f3d8e0b9baf 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | Default Java | 1.8.0_232 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/14/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/14/testReport/ | | Max. process+thread count | 4129 (vs. ulimit of 3) | | modules | C: hbase-shell . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/14/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2542: HBASE-24667 Rename configs that support atypical DNS set ups to put them in hbase.unsafe
Apache-HBase commented on pull request #2542: URL: https://github.com/apache/hbase/pull/2542#issuecomment-714820058 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 41s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 1s | master passed | | +1 :green_heart: | checkstyle | 1m 58s | master passed | | +0 :ok: | refguide | 6m 8s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | spotbugs | 3m 43s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 10s | the patch passed | | -0 :warning: | checkstyle | 0m 25s | hbase-common: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | -0 :warning: | checkstyle | 1m 14s | hbase-server: The patch generated 13 new + 41 unchanged - 1 fixed = 54 total (was 42) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +0 :ok: | refguide | 5m 52s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | hadoopcheck | 20m 9s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 4m 44s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. | | | | 63m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2542 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle refguide xml | | uname | Linux b9b80e564bbc 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/3/artifact/yetus-general-check/output/branch-site/book.html | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-common.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/3/artifact/yetus-general-check/output/patch-site/book.html | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server hbase-rest U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-714818860 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 44s | master passed | | +1 :green_heart: | compile | 1m 11s | master passed | | +1 :green_heart: | shadedjars | 7m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 44s | the patch passed | | +1 :green_heart: | compile | 1m 22s | the patch passed | | +1 :green_heart: | javac | 1m 22s | the patch passed | | +1 :green_heart: | shadedjars | 7m 37s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 219m 57s | hbase-server in the patch passed. | | | | 251m 29s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2574 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6e47103d10e7 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/2/testReport/ | | Max. process+thread count | 3272 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-714814627 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 23s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 24s | master passed | | +1 :green_heart: | compile | 1m 0s | master passed | | +1 :green_heart: | shadedjars | 7m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 49s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 21s | the patch passed | | +1 :green_heart: | compile | 1m 7s | the patch passed | | +1 :green_heart: | javac | 1m 7s | the patch passed | | +1 :green_heart: | shadedjars | 7m 34s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 206m 32s | hbase-server in the patch passed. | | | | 237m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2574 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux db8c12df3230 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/2/testReport/ | | Max. process+thread count | 3216 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
Apache-HBase commented on pull request #2483: URL: https://github.com/apache/hbase/pull/2483#issuecomment-714802738 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 29s | master passed | | +1 :green_heart: | checkstyle | 1m 25s | master passed | | +1 :green_heart: | spotbugs | 2m 31s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 24s | the patch passed | | -0 :warning: | checkstyle | 0m 14s | hbase-hadoop-compat: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | | -0 :warning: | checkstyle | 1m 11s | hbase-server: The patch generated 5 new + 277 unchanged - 0 fixed = 282 total (was 277) | | -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | hadoopcheck | 16m 58s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 48s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 42m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/16/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2483 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 8951d0391a4e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/16/artifact/yetus-general-check/output/diff-checkstyle-hbase-hadoop-compat.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/16/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | whitespace | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/16/artifact/yetus-general-check/output/whitespace-eol.txt | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/16/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bharathv commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
bharathv commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-714801212 > Also, what do you think to the idea of making the configuration setting that controls this default to TRUE on master branch and FALSE on branch-2 and branch-1 (and derived branches)? I think it's a good idea to enable it by default in the master branch. Our tests might uncover interesting edge cases in these codepaths, if any. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2542: HBASE-24667 Rename configs that support atypical DNS set ups to put them in hbase.unsafe
Apache-HBase commented on pull request #2542: URL: https://github.com/apache/hbase/pull/2542#issuecomment-714799563 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 43s | master passed | | +1 :green_heart: | compile | 1m 45s | master passed | | +1 :green_heart: | shadedjars | 6m 32s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 16s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 33s | the patch passed | | +1 :green_heart: | compile | 1m 43s | the patch passed | | +1 :green_heart: | javac | 1m 43s | the patch passed | | +1 :green_heart: | shadedjars | 6m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 13s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 33s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 158m 51s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 3m 29s | hbase-rest in the patch passed. | | | | 194m 11s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2542 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 23b36eacaebd 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/2/testReport/ | | Max. process+thread count | 4137 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server hbase-rest U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2575: HBASE-25212 [branch-2] Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2575: URL: https://github.com/apache/hbase/pull/2575#issuecomment-714797194 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 10s | branch-2 passed | | +1 :green_heart: | compile | 1m 4s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 36s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 58s | the patch passed | | +1 :green_heart: | compile | 1m 5s | the patch passed | | +1 :green_heart: | javac | 1m 5s | the patch passed | | +1 :green_heart: | shadedjars | 6m 40s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 135m 43s | hbase-server in the patch passed. | | | | 164m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2575 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1162bc37d32c 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / fdea847130 | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/2/testReport/ | | Max. process+thread count | 3532 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2542: HBASE-24667 Rename configs that support atypical DNS set ups to put them in hbase.unsafe
Apache-HBase commented on pull request #2542: URL: https://github.com/apache/hbase/pull/2542#issuecomment-714796129 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 24s | master passed | | +1 :green_heart: | compile | 2m 0s | master passed | | +1 :green_heart: | shadedjars | 6m 45s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 30s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 11s | the patch passed | | +1 :green_heart: | compile | 2m 3s | the patch passed | | +1 :green_heart: | javac | 2m 3s | the patch passed | | +1 :green_heart: | shadedjars | 7m 3s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 29s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 1s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 146m 51s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 3m 25s | hbase-rest in the patch passed. | | | | 185m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2542 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5d4ef9f4aa9d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/2/testReport/ | | Max. process+thread count | 3648 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server hbase-rest U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sandeepvinayak commented on a change in pull request #2556: HBASE-25193: Add support for row prefix and type in the WAL Pretty Printer
sandeepvinayak commented on a change in pull request #2556:
URL: https://github.com/apache/hbase/pull/2556#discussion_r510492497
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALPrettyPrinter.java
##
@@ -301,15 +318,12 @@ public void processFile(final Configuration conf, final
Path p)
List> actions = new ArrayList<>();
for (Cell cell : edit.getCells()) {
// add atomic operation to txn
- Map op = new HashMap<>(toStringMap(cell,
outputOnlyRowKey));
- if (outputValues) {
-op.put("value", Bytes.toStringBinary(CellUtil.cloneValue(cell)));
- }
- // check row output filter
- if (row == null || ((String) op.get("row")).equals(row)) {
Review comment:
@Apache9
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] sandeepvinayak commented on a change in pull request #2556: HBASE-25193: Add support for row prefix and type in the WAL Pretty Printer
sandeepvinayak commented on a change in pull request #2556:
URL: https://github.com/apache/hbase/pull/2556#discussion_r510491746
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALPrettyPrinter.java
##
@@ -301,15 +318,12 @@ public void processFile(final Configuration conf, final
Path p)
List> actions = new ArrayList<>();
for (Cell cell : edit.getCells()) {
// add atomic operation to txn
- Map op = new HashMap<>(toStringMap(cell,
outputOnlyRowKey));
- if (outputValues) {
-op.put("value", Bytes.toStringBinary(CellUtil.cloneValue(cell)));
- }
- // check row output filter
- if (row == null || ((String) op.get("row")).equals(row)) {
Review comment:
I see what you mean, I changed the flow because it was not very readable
even through it was functionally correct. Here is why I thought so:
older version
```java
Map op = new HashMap<>(toStringMap(cell,
outputOnlyRowKey));
if (outputValues) {
op.put("value", Bytes.toStringBinary(CellUtil.cloneValue(cell)));
}
// check row output filter
if (row == null || ((String) op.get("row")).equals(row)) {
actions.add(op);
}
op.put("total_size_sum", cell.heapSize());
```
Here, we are computing the stringMap and then checking if we need this row,
and then even if don't need the row we still compute `total_size_sum`. Not sure
why it was done this way but this was my concern. Hope it answers your question.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
Apache-HBase commented on pull request #2483: URL: https://github.com/apache/hbase/pull/2483#issuecomment-714785183 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 47s | master passed | | +1 :green_heart: | compile | 1m 16s | master passed | | +1 :green_heart: | shadedjars | 7m 8s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 51s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 49s | the patch passed | | +1 :green_heart: | compile | 1m 15s | the patch passed | | +1 :green_heart: | javac | 1m 15s | the patch passed | | +1 :green_heart: | shadedjars | 7m 5s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 36s | hbase-hadoop-compat in the patch passed. | | -1 :x: | unit | 209m 35s | hbase-server in the patch failed. | | | | 240m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/15/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2483 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 228b95350a2a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | Default Java | 1.8.0_232 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/15/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/15/testReport/ | | Max. process+thread count | 3680 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/15/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
Apache-HBase commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-714781691 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 40s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 10s | master passed | | +1 :green_heart: | compile | 2m 44s | master passed | | +1 :green_heart: | shadedjars | 6m 38s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 55s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 0s | the patch passed | | +1 :green_heart: | compile | 2m 44s | the patch passed | | +1 :green_heart: | javac | 2m 44s | the patch passed | | +1 :green_heart: | shadedjars | 6m 41s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 168m 44s | root in the patch passed. | | | | 205m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/14/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2308 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 0980ae25fcff 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/14/testReport/ | | Max. process+thread count | 7451 (vs. ulimit of 3) | | modules | C: hbase-shell . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/14/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-714766806 Back soon to fix the other checkstyle nits. Also, what do you think to the idea of making the configuration setting that controls this default to TRUE on master branch and FALSE on branch-2 and branch-1 (and derived branches)? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on pull request #2570: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on pull request #2570: URL: https://github.com/apache/hbase/pull/2570#issuecomment-714760677 > > Do you think "fallback to primary" logic needs to be passed down from rpc retrying caller? Then it needs to be aware of this feature and needs to maintain some state. Was trying to avoid it. Yeah, this part can be optimized later. > > If you want to make a decision based on whether the previous returned location is stale, then you need to pass something down from the rpc retrying caller to the selector, as only in rpc retrying caller we can get the exception we want. Either by adding a parameter as I proposed above to not go to secondary replicas, or as your current solution, adding a stale cache in the selector and you need to pass down the exception and the related location. > My concern here is that, the current 'simple' algorithm may not work well for some cases as my mentioned above. I suggest that we remove this part of code from the patch here(just do random selecting, without other consideration) to get this patch in quickly, and then open another issue to discuss the suitable algorithm, or maybe multiple algorithms. > Thank Duo. I think I am still not 100% clear here. Let me try to explain my understanding of what you thought before moving forward. 1. My current understanding of rpc retry caller and AsyncRegionLocator. If a location is stale, rpc retry caller will detect it and pass down exception to AsyncRegionLocator(AsyncNonMetaRegionLocator), AsyncNonMetaRegionLocator updates/cleans up table cache accordingly. 2. In the next retry, if there is no table cache entry for the key, it will call locateInMeta() to fetch location from meta. In the above flow, rpc retry caller layer is the one detecting stale location, it is up to AsyncRegionLocator layer to handle all the logic. Similarly, meta selector interface tries to do similar and build the state it needs to make decision inside itself. As for different algorithm, i.e, the case you described above, a concrete example, lets there there are 5 meta replicas, if the location from meta replica 1 stale, it may try other meta replica. We can have a different implementation of selector, AdvancedSelector, which can implement such algorithm based on the state maintained in selector. I.e, the staleCache entry has info to indicate fromMetaReplicaId. Thought that the selector interface is designed for this purpose, maybe I miss something here. The initial implementation, I kept fromMetaReplica in tableCache entry. When the patch was out for review, I dropped this change, as the current simple algorithm really does not need this info. The select interface keeps fromMetaReplicaId for future enhancement so when it is needed, at least the interface does not need to be changed. I can drop fromMetaReplicaId in the current simple implementation as it is not needed anyway. Maybe you were referring to add a flag in locateInMeta() method? If the flag says going with primary only, it will bypass all these logic and goes only to primary region so it will give upper layer more control. Please share more thoughts, thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
Apache-HBase commented on pull request #2483: URL: https://github.com/apache/hbase/pull/2483#issuecomment-714749549 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 58s | master passed | | +1 :green_heart: | compile | 1m 24s | master passed | | +1 :green_heart: | shadedjars | 6m 38s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 2s | the patch passed | | +1 :green_heart: | compile | 1m 25s | the patch passed | | +1 :green_heart: | javac | 1m 25s | the patch passed | | +1 :green_heart: | shadedjars | 6m 37s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 36s | hbase-hadoop-compat in the patch passed. | | -1 :x: | unit | 133m 56s | hbase-server in the patch failed. | | | | 164m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/15/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2483 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a1ed19608766 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | Default Java | 2020-01-14 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/15/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/15/testReport/ | | Max. process+thread count | 4602 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/15/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2575: HBASE-25212 [branch-2] Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2575: URL: https://github.com/apache/hbase/pull/2575#issuecomment-714742876 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 44s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 44s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 32s | branch-2 passed | | +1 :green_heart: | spotbugs | 3m 0s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 17s | the patch passed | | -0 :warning: | checkstyle | 1m 34s | hbase-server: The patch generated 12 new + 480 unchanged - 1 fixed = 492 total (was 481) | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 16m 13s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 56s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 45m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2575 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux f582d39508d6 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / fdea847130 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2542: HBASE-24667 Rename configs that support atypical DNS set ups to put them in hbase.unsafe
Apache-HBase commented on pull request #2542: URL: https://github.com/apache/hbase/pull/2542#issuecomment-714739401 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 54s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 11s | master passed | | +1 :green_heart: | checkstyle | 1m 53s | master passed | | +0 :ok: | refguide | 5m 48s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | spotbugs | 4m 6s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 48s | the patch passed | | -0 :warning: | checkstyle | 0m 24s | hbase-common: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | -0 :warning: | checkstyle | 1m 12s | hbase-server: The patch generated 13 new + 41 unchanged - 1 fixed = 54 total (was 42) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +0 :ok: | refguide | 5m 56s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | hadoopcheck | 20m 5s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 4m 15s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 31s | The patch does not generate ASF License warnings. | | | | 62m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2542 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle refguide xml | | uname | Linux 41b224b94494 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/2/artifact/yetus-general-check/output/branch-site/book.html | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-common.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/2/artifact/yetus-general-check/output/patch-site/book.html | | Max. process+thread count | 95 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server hbase-rest U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2542/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-714732011 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 10s | master passed | | +1 :green_heart: | checkstyle | 1m 20s | master passed | | +1 :green_heart: | spotbugs | 2m 9s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 49s | the patch passed | | -0 :warning: | checkstyle | 1m 16s | hbase-server: The patch generated 9 new + 410 unchanged - 1 fixed = 419 total (was 411) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 55s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 20s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 43m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2574 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux aa2ced3aef11 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2570: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on a change in pull request #2570:
URL: https://github.com/apache/hbase/pull/2570#discussion_r510415238
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestMetaRegionReplicaReplicationEndpoint.java
##
@@ -63,17 +68,20 @@
/**
* Tests RegionReplicaReplicationEndpoint class for hbase:meta by setting up
region replicas and
* verifying async wal replication replays the edits to the secondary region
in various scenarios.
+ *
* @see TestRegionReplicaReplicationEndpoint
*/
@Category({LargeTests.class})
Review comment:
will add "RegionServerTests" tag.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2570: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on a change in pull request #2570:
URL: https://github.com/apache/hbase/pull/2570#discussion_r510414632
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncNonMetaRegionLocatorWithMetaReplicaLoadBalance.java
##
@@ -0,0 +1,84 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.client;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.security.User;
+import org.apache.hadoop.hbase.testclassification.ClientTests;
+import org.apache.hadoop.hbase.testclassification.MediumTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.ServerRegionReplicaUtil;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.experimental.categories.Category;
+
+@Category({ MediumTests.class, ClientTests.class })
+public class TestAsyncNonMetaRegionLocatorWithMetaReplicaLoadBalance
Review comment:
Let me check it. Trying to make sure that with Parameterized test,
minicluster is going to be recreated (once I was there, but forgot the details).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2570: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on a change in pull request #2570: URL: https://github.com/apache/hbase/pull/2570#discussion_r510413345 ## File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java ## @@ -198,6 +208,36 @@ private boolean tryComplete(LocateRequest req, CompletableFuture
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2570: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on a change in pull request #2570: URL: https://github.com/apache/hbase/pull/2570#discussion_r510413507 ## File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java ## @@ -198,6 +208,36 @@ private boolean tryComplete(LocateRequest req, CompletableFuture
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2570: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on a change in pull request #2570:
URL: https://github.com/apache/hbase/pull/2570#discussion_r510412906
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
##
@@ -392,6 +406,16 @@ public Connection toConnection() {
return c;
}
+ private boolean isMetaReplicaLBMode() {
+if (conf.getBoolean(USE_META_REPLICAS, DEFAULT_USE_META_REPLICAS)) {
+ String metaReplicaMode = conf.get(META_REPLICAS_MODE, "");
+ if (MetaReplicaMode.LoadBalance.toString().equals(metaReplicaMode)) {
Review comment:
Good catch! I think it should be case-insensitive.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2570: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on a change in pull request #2570:
URL: https://github.com/apache/hbase/pull/2570#discussion_r510412431
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetaReplicaLoadBalanceReplicaSimpleChooser.java
##
@@ -0,0 +1,248 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.client;
+
+import org.apache.commons.lang3.builder.ToStringBuilder;
+import org.apache.hadoop.hbase.HRegionLocation;
+import org.apache.hadoop.hbase.ScheduledChore;
+import org.apache.hadoop.hbase.Stoppable;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import java.util.Iterator;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentNavigableMap;
+import java.util.concurrent.ConcurrentSkipListMap;
+import java.util.concurrent.ThreadLocalRandom;
+import static org.apache.hadoop.hbase.HConstants.DEFAULT_META_REPLICA_NUM;
+import static org.apache.hadoop.hbase.HConstants.META_REPLICAS_NUM;
+import static org.apache.hadoop.hbase.client.ConnectionUtils.isEmptyStopRow;
+import static org.apache.hadoop.hbase.util.Bytes.BYTES_COMPARATOR;
+import static org.apache.hadoop.hbase.util.ConcurrentMapUtils.computeIfAbsent;
+
+/**
+ * MetaReplicaLoadBalanceReplicaSimpleChooser implements a simple meta replica
load balancing
+ * algorithm. It maintains a stale location cache for each table. Whenever
client looks up meta,
+ * it first check if the row is the stale location cache, if yes, this means
the the location from
+ * meta replica is stale, it will go to the primary meta to look up
update-to-date location;
+ * otherwise, it will randomly pick up a meta replica region for meta lookup.
When clients receive
+ * RegionNotServedException from region servers, it will add these region
locations to the stale
+ * location cache. The stale cache will be cleaned up periodically by a chore.
+ */
+
+/**
+ * StaleLocationCacheEntry is the entry when a stale location is reported by
an client.
+ */
+class StaleLocationCacheEntry {
+ // meta replica id where
+ private int metaReplicaId;
+
+ // timestamp in milliseconds
+ private long timestamp;
+
+ private byte[] endKey;
+
+ StaleLocationCacheEntry(final int metaReplicaId, final byte[] endKey) {
+this.metaReplicaId = metaReplicaId;
+this.endKey = endKey;
+timestamp = System.currentTimeMillis();
+ }
+
+ public byte[] getEndKey() {
+return this.endKey;
+ }
+
+ public int getMetaReplicaId() {
+return this.metaReplicaId;
+ }
+ public long getTimestamp() {
+return this.timestamp;
+ }
+
+ @Override
+ public String toString() {
+return new ToStringBuilder(this)
+ .append("endKey", endKey)
+ .append("metaReplicaId", metaReplicaId)
+ .append("timestamp", timestamp)
+ .toString();
+ }
+}
+
+/**
+ * A simple implementation of MetaReplicaLoadBalanceReplicaChooser.
+ *
+ * It follows a simple algorithm to choose a meta replica to go:
+ *
+ * 1. If there is no stale location entry for rows it looks up, it will
randomly
+ * pick a meta replica region to do lookup.
+ * 2. If the location from meta replica region is stale, client gets
RegionNotServedException
+ * from region server, in this case, it will create
StaleLocationCacheEntry in
+ * MetaReplicaLoadBalanceReplicaSimpleChooser.
+ * 3. When client tries to do meta lookup, it checks StaleLocationCache first
for rows it tries to
+ * lookup, if entry exists, it will go with primary meta region to do
lookup; otherwise, it
+ * will follow step 1.
+ * 4. A chore will periodically run to clean up cache entries in the
StaleLocationCache.
+ */
+class MetaReplicaLoadBalanceReplicaSimpleChooser implements
MetaReplicaLoadBalanceReplicaChooser {
Review comment:
Will do.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service,
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2570: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on a change in pull request #2570:
URL: https://github.com/apache/hbase/pull/2570#discussion_r510412184
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetaReplicaLoadBalanceReplicaSimpleChooser.java
##
@@ -0,0 +1,248 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.client;
+
+import org.apache.commons.lang3.builder.ToStringBuilder;
+import org.apache.hadoop.hbase.HRegionLocation;
+import org.apache.hadoop.hbase.ScheduledChore;
+import org.apache.hadoop.hbase.Stoppable;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import java.util.Iterator;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentNavigableMap;
+import java.util.concurrent.ConcurrentSkipListMap;
+import java.util.concurrent.ThreadLocalRandom;
+import static org.apache.hadoop.hbase.HConstants.DEFAULT_META_REPLICA_NUM;
+import static org.apache.hadoop.hbase.HConstants.META_REPLICAS_NUM;
+import static org.apache.hadoop.hbase.client.ConnectionUtils.isEmptyStopRow;
+import static org.apache.hadoop.hbase.util.Bytes.BYTES_COMPARATOR;
+import static org.apache.hadoop.hbase.util.ConcurrentMapUtils.computeIfAbsent;
+
+/**
+ * MetaReplicaLoadBalanceReplicaSimpleChooser implements a simple meta replica
load balancing
+ * algorithm. It maintains a stale location cache for each table. Whenever
client looks up meta,
+ * it first check if the row is the stale location cache, if yes, this means
the the location from
+ * meta replica is stale, it will go to the primary meta to look up
update-to-date location;
+ * otherwise, it will randomly pick up a meta replica region for meta lookup.
When clients receive
+ * RegionNotServedException from region servers, it will add these region
locations to the stale
+ * location cache. The stale cache will be cleaned up periodically by a chore.
+ */
+
+/**
+ * StaleLocationCacheEntry is the entry when a stale location is reported by
an client.
+ */
+class StaleLocationCacheEntry {
Review comment:
Got it, will make it an inner class.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2570: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on a change in pull request #2570: URL: https://github.com/apache/hbase/pull/2570#discussion_r510411776 ## File path: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java ## @@ -1128,6 +1128,9 @@ /** Conf key for enabling meta replication */ public static final String USE_META_REPLICAS = "hbase.meta.replicas.use"; + public static final String META_REPLICAS_MODE = "hbase.meta.replicas.mode"; Review comment: "hbase.catalog.replicas.mode"? Or we want to include "meta" here so it can only enable meta table. In the future, let's say root table needs to be supported, so another config knob will be added. By this way, configs are separated for both for current "meta" and future system tables. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2570: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on a change in pull request #2570:
URL: https://github.com/apache/hbase/pull/2570#discussion_r510410275
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetaReplicaLoadBalanceReplicaChooser.java
##
@@ -0,0 +1,53 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.client;
+
+import org.apache.hadoop.hbase.HRegionLocation;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * There are two modes with meta replica support.
+ * HighAvailable- Client sends requests to the primary meta region
first, within a
+ * configured amount of time, if there is no response
coming back,
+ * client sends requests to all replica regions and takes
the first
+ * response.
+ *
+ * LoadBalance - Client sends requests to meta replica regions in a
round-robin mode,
+ * if results from replica regions are stale, next time,
client sends requests for
+ * these stable locations to the primary meta region. In
this mode, scan
+ * requests are load balanced across all replica regions.
+ */
+enum MetaReplicaMode {
+ None,
+ HighAvailable,
+ LoadBalance
+}
+
+/**
+ * A Meta replica chooser decides which meta replica to go for scan requests.
+ */
+@InterfaceAudience.Private
+public interface MetaReplicaLoadBalanceReplicaChooser {
+
+ void updateCacheOnError(final HRegionLocation loc, final int
fromMetaReplicaId);
Review comment:
Agreed. I am going to rename it "Catalog" instead of "Meta", to reflect
the fact that it can be used to location lookup service. As for user tables, if
LoadBalance mode needs to be supported, current interface already supports it.
It is up to application to support this mode.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] gkanade commented on a change in pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
gkanade commented on a change in pull request #2483: URL: https://github.com/apache/hbase/pull/2483#discussion_r510389863 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java ## @@ -3075,6 +3080,13 @@ private RegionScannerImpl getScanner(Scan scan, List additional checkFamily(family); } } + if((Bytes.equals(scan.getStartRow(), getRegionInfo().getStartKey()) Review comment: done, now it is at time scanner instantiated only. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] gkanade commented on a change in pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
gkanade commented on a change in pull request #2483:
URL: https://github.com/apache/hbase/pull/2483#discussion_r510383502
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -7699,6 +7710,14 @@ public void run() throws IOException {
// callback
this.close();
}
+
+public boolean isFullRegionScan() {
Review comment:
No longer a method, added check directly before scanner instantiation
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] gkanade commented on a change in pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
gkanade commented on a change in pull request #2483:
URL: https://github.com/apache/hbase/pull/2483#discussion_r510383316
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -7238,6 +7246,9 @@ public boolean nextRaw(List outResults,
ScannerContext scannerContext)
rsServices.getMetrics().updateReadQueryMeter(getRegionInfo().getTable());
}
+ if(isFullRegionScan()) {
Review comment:
Remove now
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] gkanade commented on a change in pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
gkanade commented on a change in pull request #2483:
URL: https://github.com/apache/hbase/pull/2483#discussion_r510383013
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -9143,4 +9162,8 @@ public void setReadRequestsCount(long readRequestsCount) {
public void setWriteRequestsCount(long writeRequestsCount) {
this.writeRequestsCount.add(writeRequestsCount);
}
+
+ public void setFullScanRequestsCount(long fullScanRequestsCount) {
Review comment:
Removed
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java
##
@@ -23,6 +23,7 @@
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseInterfaceAudience;
+import org.apache.hadoop.hbase.HConstants;
Review comment:
Removed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] gkanade commented on a change in pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
gkanade commented on a change in pull request #2483: URL: https://github.com/apache/hbase/pull/2483#discussion_r510382742 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java ## @@ -115,4 +116,6 @@ */ boolean nextRaw(List result, ScannerContext scannerContext) throws IOException; + + boolean isFullRegionScan(); Review comment: Moved metric accounting to HRegion layer now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
Apache-HBase commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-714690375 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for patch | | -0 :warning: | rubocop | 0m 5s | The patch generated 8 new + 24 unchanged - 1 fixed = 32 total (was 25) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 16m 53s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 28s | The patch does not generate ASF License warnings. | | | | 25m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/14/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2308 | | Optional Tests | dupname asflicense rubocop hadoopcheck xml | | uname | Linux 701a5bb095fe 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | rubocop | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/14/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | Max. process+thread count | 76 (vs. ulimit of 3) | | modules | C: hbase-shell . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/14/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) rubocop=0.80.0 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] gkanade commented on a change in pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
gkanade commented on a change in pull request #2483:
URL: https://github.com/apache/hbase/pull/2483#discussion_r510381778
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
##
@@ -3498,6 +3501,18 @@ public ScanResponse scan(final RpcController controller,
final ScanRequest reque
throw new ServiceException(e);
}
HRegion region = rsh.r;
+// If the start row is the actual start Row of the region and end row is
actual end row of region
+// or
+// If the stop is not specified or the stop Row is greater than the end
key of the current region
+if ((Bytes.equals(rsh.s.getStartRow(),
region.getRegionInfo().getStartKey())
+ && Bytes.equals(rsh.s.getStopRow(), region.getRegionInfo().getEndKey()))
+ || Bytes.equals(rsh.s.getStopRow(), HConstants.EMPTY_END_ROW)
+ || (Bytes.compareTo(region.getRegionInfo().getEndKey(),
rsh.s.getStopRow()) <= 0
+ && !Bytes.equals(region.getRegionInfo().getEndKey(),
HConstants.EMPTY_END_ROW))) {
+ if(!region.getTableDescriptor().isMetaTable()) {
Review comment:
This is now moved to HRegion layer. We check it once when we instantiate
the scanner.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2483: HBASE-25026 - Create a metric to track scans that have no start row and/or stop row
Apache-HBase commented on pull request #2483: URL: https://github.com/apache/hbase/pull/2483#issuecomment-714686601 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 52s | master passed | | +1 :green_heart: | checkstyle | 1m 29s | master passed | | +1 :green_heart: | spotbugs | 2m 40s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 47s | the patch passed | | -0 :warning: | checkstyle | 0m 12s | hbase-hadoop-compat: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | | -0 :warning: | checkstyle | 1m 17s | hbase-server: The patch generated 5 new + 283 unchanged - 0 fixed = 288 total (was 283) | | -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | hadoopcheck | 18m 54s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 21s | The patch does not generate ASF License warnings. | | | | 45m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/15/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2483 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 83f107020a60 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/15/artifact/yetus-general-check/output/diff-checkstyle-hbase-hadoop-compat.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/15/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | whitespace | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/15/artifact/yetus-general-check/output/whitespace-eol.txt | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2483/15/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25212) Optionally abort requests in progress after deciding a region should close
[
https://issues.apache.org/jira/browse/HBASE-25212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219261#comment-17219261
]
Andrew Kyle Purtell commented on HBASE-25212:
-
After upgrading to Java 11 realized I was just missing a change to HTU. Never
mind.
> Optionally abort requests in progress after deciding a region should close
> --
>
> Key: HBASE-25212
> URL: https://issues.apache.org/jira/browse/HBASE-25212
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0
>
>
> After deciding a region should be closed, the regionserver will set the
> internal region state to closing and wait for all pending requests to
> complete, via a rendezvous on the region lock. In closing state the region
> will not accept any new requests but requests in progress will be allowed to
> complete before the close action takes place. In our production we see
> outlier wait times on this lock in excess of several minutes.
> During close when there are requests in flight the regionserver is subject to
> any conceivable reason for delay, like full scans over large regions,
> expensive filtering hierarchies, bugs, or store level performance problems
> like slow HDFS. The regionserver should interrupt requests in progress to
> facilitate smaller/shorter close times on an opt-in basis.
> Optionally, via configuration parameter -- which would be a system wide
> default set in hbase-site.xml in common practice but could be overridden in
> table schema for per table settings -- interrupt requests in progress holding
> the region lock rather than wait for completion of all operations in flight.
> Send back NotServingRegionException("region is closing") to the clients of
> the interrupted operations, like we do after the write lock is acquired. The
> client will transparently relocate the region data and resubmit the aborted
> requests per normal retry policy. This can be less disruptive than waiting
> for very long times for a region to close in extreme outlier cases (e.g. 50
> minutes). In such extreme cases it is better to abort the regionserver if the
> close lock cannot be acquired in a reasonable amount of time, because the
> region cannot be made available again until it has closed.
> After waiting for all requests to complete then we flush the region's
> memstore and finish the close. The flush portion of the close process is out
> of scope of this proposal. Under normal conditions the flush portion of the
> close completes quickly. It is specifically waits on the close lock that has
> been an occasional issue in our production that causes difficulty achieving
> 99.99% availability.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] apurtell removed a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell removed a comment on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-714639693 The unit test failures are interesting. These new test work reliably with Java 8, which is what I develop with and use in production. They do not appear to work on Java 11 because of a timing or some other difference, which I do not use for development. I will upgrade my dev environment to use Java 11 and improve the tests as needed but it does not reflect what we use to build or deploy to production. I wonder if others face this issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-714676191 Deleted comment about JDK version. After upgrading to Java 11 realized I was just missing a change to HTU. Never mind. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2539: HBASE-25181 Configure hash algorithm in wrapped encryption keys
Apache-HBase commented on pull request #2539: URL: https://github.com/apache/hbase/pull/2539#issuecomment-714673473 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 3s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 43s | master passed | | +1 :green_heart: | checkstyle | 1m 15s | master passed | | +1 :green_heart: | spotbugs | 5m 17s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 24s | the patch passed | | +1 :green_heart: | checkstyle | 0m 10s | The patch passed checkstyle in hbase-protocol-shaded | | -0 :warning: | checkstyle | 0m 25s | hbase-common: The patch generated 1 new + 52 unchanged - 0 fixed = 53 total (was 52) | | +1 :green_heart: | checkstyle | 0m 27s | hbase-client: The patch generated 0 new + 3 unchanged - 2 fixed = 3 total (was 5) | | +1 :green_heart: | checkstyle | 0m 12s | The patch passed checkstyle in hbase-shell | | +1 :green_heart: | rubocop | 0m 7s | There were no new rubocop issues. | | -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | hadoopcheck | 16m 59s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | hbaseprotoc | 2m 10s | the patch passed | | +1 :green_heart: | spotbugs | 5m 42s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 48s | The patch does not generate ASF License warnings. | | | | 50m 16s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2539/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2539 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle cc hbaseprotoc prototool rubocop | | uname | Linux 7d73e176d94c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2539/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-common.txt | | whitespace | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2539/3/artifact/yetus-general-check/output/whitespace-eol.txt | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2539/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 rubocop=0.80.0 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2539: HBASE-25181 Configure hash algorithm in wrapped encryption keys
Apache-HBase commented on pull request #2539: URL: https://github.com/apache/hbase/pull/2539#issuecomment-714669413 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 4s | master passed | | +1 :green_heart: | compile | 2m 14s | master passed | | +1 :green_heart: | shadedjars | 6m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 19s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 2s | the patch passed | | +1 :green_heart: | compile | 2m 16s | the patch passed | | +1 :green_heart: | javac | 2m 16s | the patch passed | | +1 :green_heart: | shadedjars | 6m 36s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 18s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 0s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 41s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 7s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 6m 57s | hbase-shell in the patch passed. | | | | 42m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2539/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2539 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ab69482296e8 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2539/3/testReport/ | | Max. process+thread count | 2342 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2539/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2539: HBASE-25181 Configure hash algorithm in wrapped encryption keys
Apache-HBase commented on pull request #2539: URL: https://github.com/apache/hbase/pull/2539#issuecomment-714668995 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 16s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 7s | master passed | | +1 :green_heart: | compile | 1m 53s | master passed | | +1 :green_heart: | shadedjars | 7m 8s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 6s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 54s | the patch passed | | +1 :green_heart: | compile | 1m 55s | the patch passed | | +1 :green_heart: | javac | 1m 55s | the patch passed | | +1 :green_heart: | shadedjars | 7m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 48s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 31s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 12s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 6m 57s | hbase-shell in the patch passed. | | | | 42m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2539/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2539 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 094a9fc560cc 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 74df1e1aba | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2539/3/testReport/ | | Max. process+thread count | 1538 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2539/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25186) TestMasterRegionOnTwoFileSystems is failing after HBASE-25065
[
https://issues.apache.org/jira/browse/HBASE-25186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219233#comment-17219233
]
Hudson commented on HBASE-25186:
Results for branch branch-2
[build #81 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/81/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/81/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/81/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/81/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/81/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestMasterRegionOnTwoFileSystems is failing after HBASE-25065
> -
>
> Key: HBASE-25186
> URL: https://issues.apache.org/jira/browse/HBASE-25186
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 3.0.0-alpha-1, 2.4.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> After HBASE-25065, we are having a test case failure with
> TestMasterRegionOnTwoFileSystems.
> The reason is that we manually trigger a WAL roll on the master region. As
> part of the WAL roll we expect the Master region's WAL will also be moved
> from region oldWAL dir to the global oldWAL directory. This happens after
> afterRoll() method in AbstractWALRoller.
> Since now the WAL archival is asynchronous the afterRoll() method does not
> find any WAL file to be moved in the local region oldWAL dir. So the movement
> to global oldWAL dir does not happen.
> The test case checks for the file in the oldWAL dir and since it is not found
> the test timesout. WE need a way to fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] symat commented on pull request #2539: HBASE-25181 Configure hash algorithm in wrapped encryption keys
symat commented on pull request #2539: URL: https://github.com/apache/hbase/pull/2539#issuecomment-714647401 I made this behaviour configurable, just pushed the change. (I also added new tests for it) What do you think? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
Apache-HBase commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-714644766 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 41s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 33s | master passed | | +1 :green_heart: | compile | 2m 18s | master passed | | +1 :green_heart: | shadedjars | 6m 31s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 15s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 27s | the patch passed | | +1 :green_heart: | compile | 2m 22s | the patch passed | | +1 :green_heart: | javac | 2m 22s | the patch passed | | +1 :green_heart: | shadedjars | 6m 29s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 21s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 326m 11s | root in the patch failed. | | | | 359m 28s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/13/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2308 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 830c1bf8e873 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 2cd5f914af | | Default Java | 1.8.0_232 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/13/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/13/testReport/ | | Max. process+thread count | 5783 (vs. ulimit of 3) | | modules | C: hbase-shell . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/13/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell edited a comment on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-714639693 The unit test failures are interesting. These new test work reliably with Java 8, which is what I develop with and use in production. They do not appear to work on Java 11 because of a timing or some other difference, which I do not use for development. I will upgrade my dev environment to use Java 11 and improve the tests as needed but it does not reflect what we use to build or deploy to production. I wonder if others face this issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2570: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on a change in pull request #2570:
URL: https://github.com/apache/hbase/pull/2570#discussion_r510332100
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -435,6 +475,14 @@ private void locateInMeta(TableName tableName,
LocateRequest req) {
.setReadType(ReadType.PREAD);
if (useMetaReplicas) {
scan.setConsistency(Consistency.TIMELINE);
+
+ // If it is meta replica LoadBalance mode, pick up a meta replica to
scan. It will try to
+ // pick up meta replica region to scan, in certain error cases, it may
pick primary meta
+ // region to scan.
+ if (isMetaReplicaLBMode()) {
+scan.setReplicaId(this.metaReplicaChooser.chooseReplicaToGo(tableName,
Review comment:
addressed in the next patch.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-25212) Optionally abort requests in progress after deciding a region should close
[
https://issues.apache.org/jira/browse/HBASE-25212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219207#comment-17219207
]
Andrew Kyle Purtell commented on HBASE-25212:
-
Unit test failures reported on the master PR are interesting. These new test
work reliably with Java 8, which is what I develop with and use in production.
They do not appear to work on Java 11 because of a timing or some other
difference, which I do not use for development. I will upgrade my dev
environment to use Java 11 but it does not reflect what we use to build or
deploy to production. I wonder if others face this issue.
> Optionally abort requests in progress after deciding a region should close
> --
>
> Key: HBASE-25212
> URL: https://issues.apache.org/jira/browse/HBASE-25212
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0
>
>
> After deciding a region should be closed, the regionserver will set the
> internal region state to closing and wait for all pending requests to
> complete, via a rendezvous on the region lock. In closing state the region
> will not accept any new requests but requests in progress will be allowed to
> complete before the close action takes place. In our production we see
> outlier wait times on this lock in excess of several minutes.
> During close when there are requests in flight the regionserver is subject to
> any conceivable reason for delay, like full scans over large regions,
> expensive filtering hierarchies, bugs, or store level performance problems
> like slow HDFS. The regionserver should interrupt requests in progress to
> facilitate smaller/shorter close times on an opt-in basis.
> Optionally, via configuration parameter -- which would be a system wide
> default set in hbase-site.xml in common practice but could be overridden in
> table schema for per table settings -- interrupt requests in progress holding
> the region lock rather than wait for completion of all operations in flight.
> Send back NotServingRegionException("region is closing") to the clients of
> the interrupted operations, like we do after the write lock is acquired. The
> client will transparently relocate the region data and resubmit the aborted
> requests per normal retry policy. This can be less disruptive than waiting
> for very long times for a region to close in extreme outlier cases (e.g. 50
> minutes). In such extreme cases it is better to abort the regionserver if the
> close lock cannot be acquired in a reasonable amount of time, because the
> region cannot be made available again until it has closed.
> After waiting for all requests to complete then we flush the region's
> memstore and finish the close. The flush portion of the close process is out
> of scope of this proposal. Under normal conditions the flush portion of the
> close completes quickly. It is specifically waits on the close lock that has
> been an occasional issue in our production that causes difficulty achieving
> 99.99% availability.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] apurtell edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell edited a comment on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-714639693 The unit test failures are interesting. These new test work reliably with Java 8, which is what I develop with and use in production. They do not appear to work on Java 11 because of a timing or some other difference, which I do not use for development. I will upgrade my dev environment to use Java 11 but it does not reflect what we use to build or deploy to production. I wonder if others face this issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-714639693 The unit test failures are interesting. These new test work reliably with Java 8, which is what I develop with and use in production. They do not appear to work on Java 11 because of a timing or some other difference, which I do not use for development. I will upgrade my dev environment to use Java 11 but as it does not reflect what we use to build or deploy to production. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-714636166 I see findbugs doesn't like my deliberate fall through in switch statements. In old school programming we'd do this intentionally because we think we are clever. Anyway will fix. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r510320642
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionInterrupt.java
##
@@ -0,0 +1,357 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver;
+
+import static org.junit.Assert.assertTrue;
+
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.util.Optional;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.NotServingRegionException;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.Waiter;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Append;
+import org.apache.hadoop.hbase.client.BufferedMutator;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Durability;
+import org.apache.hadoop.hbase.client.Increment;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.coprocessor.ObserverContext;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessor;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
+import org.apache.hadoop.hbase.coprocessor.RegionObserver;
+import org.apache.hadoop.hbase.exceptions.DeserializationException;
+import org.apache.hadoop.hbase.filter.FilterBase;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.testclassification.RegionServerTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.rules.TestName;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+@Category({RegionServerTests.class, LargeTests.class})
+public class TestRegionInterrupt {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+HBaseClassTestRule.forClass(TestRegionInterrupt.class);
+
+ private static HBaseTestingUtility TEST_UTIL = new HBaseTestingUtility();
+ private static final Logger LOG =
LoggerFactory.getLogger(TestRegionInterrupt.class);
+
+ static final int SLEEP_TIME = 10 * 1000;
+ static final byte[] FAMILY = Bytes.toBytes("info");
+
+ @Rule
+ public TestName name = new TestName();
Review comment:
Did not know about this, ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r510320411
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
##
@@ -7364,4 +7367,154 @@ protected HStoreForTesting(final HRegion region,
return super.doCompaction(cr, filesToCompact, user, compactionStartTime,
newFiles);
}
}
+
+ @Test(timeout=2)
+ public void testCloseNoInterrupt() throws Exception {
+byte[] cf1 = Bytes.toBytes("CF1");
+byte[][] families = { cf1 };
+
+Configuration conf = HBaseConfiguration.create();
Review comment:
Ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574: URL: https://github.com/apache/hbase/pull/2574#discussion_r510320339 ## File path: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java ## @@ -7364,4 +7367,154 @@ protected HStoreForTesting(final HRegion region, return super.doCompaction(cr, filesToCompact, user, compactionStartTime, newFiles); } } + + @Test(timeout=2) Review comment: Ok This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r510319462
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -6588,8 +6677,10 @@ protected RowLock getRowLockInternal(byte[] row, boolean
readLock, final RowLock
success = true;
return result;
} catch (InterruptedException ie) {
- LOG.warn("Thread interrupted waiting for lock on row: {}, in region {}",
rowKey,
-getRegionInfo().getRegionNameAsString());
+ if (LOG.isDebugEnabled()) {
+LOG.debug("Thread interrupted waiting for lock on row: {}, in region
{}", rowKey,
Review comment:
This was an existing case where the client would get back an IIOE. I
didn't change it so the current semantics are preserved. It can be changed to
use throwOnInterrupt
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r510319462
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -6588,8 +6677,10 @@ protected RowLock getRowLockInternal(byte[] row, boolean
readLock, final RowLock
success = true;
return result;
} catch (InterruptedException ie) {
- LOG.warn("Thread interrupted waiting for lock on row: {}, in region {}",
rowKey,
-getRegionInfo().getRegionNameAsString());
+ if (LOG.isDebugEnabled()) {
+LOG.debug("Thread interrupted waiting for lock on row: {}, in region
{}", rowKey,
Review comment:
This was an existing case where the client would get back an IIOE. I
didn't change it so the current semantics are preserved. It can be changed to
use throwIfInterrupted
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r510317299
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -4550,6 +4619,10 @@ private void doMiniBatchMutate(BatchOperation
batchOp) throws IOException {
/** Keep track of the locks we hold so we can release them in finally
clause */
List acquiredRowLocks =
Lists.newArrayListWithCapacity(batchOp.size());
try {
+ // Check for thread interrupt status in case we have been signaled from
+ // #interruptRegionOperation.
+ checkInterrupt();
Review comment:
Yes, but on the other hand we are basically checking interrupt status
manually, as one has to do in Java, so can check and exit early at only what we
think are safe points, which is nice.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r510317538
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -8748,6 +8863,11 @@ public void startRegionOperation(Operation op) throws
IOException {
throw new
NotServingRegionException(getRegionInfo().getRegionNameAsString() + " is
closing");
}
lock(lock.readLock());
+// Update regionLockHolders ONLY for any startRegionOperation call that is
invoked from an RPC handler
+Thread thisThread = Thread.currentThread();
+if (isInterruptableOp) {
+ regionLockHolders.put(thisThread.hashCode(), thisThread);
Review comment:
Ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r510317299
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -4550,6 +4619,10 @@ private void doMiniBatchMutate(BatchOperation
batchOp) throws IOException {
/** Keep track of the locks we hold so we can release them in finally
clause */
List acquiredRowLocks =
Lists.newArrayListWithCapacity(batchOp.size());
try {
+ // Check for thread interrupt status in case we have been signaled from
+ // #interruptRegionOperation.
+ checkInterrupt();
Review comment:
Yes, but on the other hand we are basically checking interrupt status
manually, as one has to do in Java, so can check and exit early only at only
what we think are safe points.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r510316643
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -1174,7 +1178,7 @@ public HStore call() throws IOException {
LOG.info("Setting FlushNonSloppyStoresFirstPolicy for the region=" +
this);
}
} catch (InterruptedException e) {
-throw (InterruptedIOException)new
InterruptedIOException().initCause(e);
+throwOnInterrupt(e);
Review comment:
Ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] symat commented on pull request #2539: HBASE-25181 Configure hash algorithm in wrapped encryption keys
symat commented on pull request #2539: URL: https://github.com/apache/hbase/pull/2539#issuecomment-714597486 >We should use the one in the hfile, otherwise the only way to change the config is to throw away your cluster and start over. That is a good point... I'll change accordingly. > I think a DEBUG message of what hash we're using is a good idea. I don't think it needs to be at a more visible level since most uses wouldn't be impacted by that information. Honestly, I wouldn't even log this, only if the hash algorithm in the HFile mismatches with the one being configured. (normally you can expect to have the HFiles the correct / configured algorithm) > For something like a restricted system that shouldn't use unapproved algorithms can we rely on an external enforcement mechanism? well, it depends on the JVM configuration. FIPS compliant security providers (at least the one that I know) allows you to configure "strict mode" via JVM properties, which will throw exception when you try using non compliant algorithm. But you can not really enable this mode for HBase, as we still use MD5 e.g. for region name encoding. Using MD5 in non-cryptographic / non-security context is actually fine, at least for FIPS compliance. But it does prevent the users to enable this "strict mode". So I think at least showing a more visible sign (e.g. a warning message) would be beneficial for these customers. Alternatively we could even make this behaviour configurable (like introducing a config `hbase.crypto.key.hash.algorithm.failOnMismatch`false by default). Then if someone really changes this algorithm parameter for regulations, then she/he can choose to be more strict and be less backward-compatible. I tend towards this approach, even if it means yet an other HBase parameter in addition to the zillion we already have. :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] wchevreuil commented on a change in pull request #2579: HBASE-24999 Master manages ReplicationServers
wchevreuil commented on a change in pull request #2579:
URL: https://github.com/apache/hbase/pull/2579#discussion_r510244366
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HBaseReplicationEndpoint.java
##
@@ -288,39 +258,26 @@ public boolean isAborted() {
return false;
}
- /**
- * Get the connection to peer cluster
- * @return connection to peer cluster
- * @throws IOException If anything goes wrong connecting
- */
- private synchronized AsyncClusterConnection getPeerConnection() throws
IOException {
-if (peerConnection == null) {
- Configuration conf = ctx.getConfiguration();
- peerConnection =
ClusterConnectionFactory.createAsyncClusterConnection(conf, null,
- UserProvider.instantiate(conf).getCurrent());
-}
-return peerConnection;
- }
-
/**
* Get the list of all the servers that are responsible for replication sink
* from the specified peer master
* @return list of server addresses or an empty list if the slave is
unavailable
Review comment:
Is it true that an empty list would be returned if we can't reach slave
master? Looks like we either throw an IOException or a RemoteException.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ReplicationServerManager.java
##
@@ -0,0 +1,199 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentNavigableMap;
+import java.util.concurrent.ConcurrentSkipListMap;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.ScheduledChore;
+import org.apache.hadoop.hbase.ServerMetrics;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * The ServerManager class manages info about replication servers.
+ *
+ * Maintains lists of online and dead servers.
+ *
+ * Servers are distinguished in two different ways. A given server has a
+ * location, specified by hostname and port, and of which there can only be one
+ * online at any given time. A server instance is specified by the location
+ * (hostname and port) as well as the startcode (timestamp from when the server
+ * was started). This is used to differentiate a restarted instance of a given
+ * server from the original instance.
+ */
+@InterfaceAudience.Private
+public class ReplicationServerManager {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(ReplicationServerManager.class);
+
+ public static final String ONLINE_SERVER_REFRESH_INTERVAL =
+ "hbase.master.replication.server.refresh.interval";
+ public static final int ONLINE_SERVER_REFRESH_INTERVAL_DEFAULT = 60 * 1000;
// 1 mins
+
+ private final MasterServices master;
+
+ /** Map of registered servers to their current load */
+ private final ConcurrentNavigableMap
onlineServers =
+new ConcurrentSkipListMap<>();
+
+ private OnlineServerRefresher onlineServerRefresher;
+ private int refreshPeriod;
+
+ /**
+ * Constructor.
+ */
+ public ReplicationServerManager(final MasterServices master) {
+this.master = master;
+ }
+
+ /**
+ * start chore in ServerManager
+ */
+ public void startChore() {
+Configuration conf = master.getConfiguration();
+refreshPeriod = conf.getInt(ONLINE_SERVER_REFRESH_INTERVAL,
+ONLINE_SERVER_REFRESH_INTERVAL_DEFAULT);
+onlineServerRefresher = new
OnlineServerRefresher("ReplicationServerRefresher", refreshPeriod);
+master.getChoreService().scheduleChore(onlineServerRefresher);
+ }
+
+ /**
+ * Stop the ServerManager.
+ */
+ public void stop() {
+if (onlineServerRefresher != null) {
+ onlineServerRefresher.cancel();
+}
+ }
+
+ public void serverReport(ServerName sn, ServerMetrics sl) {
+if (null == this.onlineServers.replace(sn, sl)) {
+ if (!checkAndRecordNewServer(sn, sl)) {
+LOG.info("ReplicationServerReport ignored, could not record the
server: {}", sn);
+ }
+}
+ }
+
+ /**
+
[jira] [Comment Edited] (HBASE-25090) CompactionConfiguration logs unrealistic store file sizes
[
https://issues.apache.org/jira/browse/HBASE-25090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219100#comment-17219100
]
Zhuoyue Huang edited comment on HBASE-25090 at 10/22/20, 3:30 PM:
--
I guess this is not a bug.
{code:java}
size [128 MB, 8.00 EB, 8.00 EB); {code}
represents [*minCompactSize*, *maxCompactSize*, *offPeakMaxCompactSize*).
The default value of *maxCompactSize(hbase.hstore.compaction.max.size*) and
*offPeakMaxCompactSize(hbase.hstore.compaction.max.size.offpeak*) is
*Long.MAX*, so the display is 8.00EB.
But the information here is easy to misunderstand. How about changing it like
this?
{code:java}
size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00
EB); files [minFilesToCompact:3, maxFilesToCompact:10);
{code}
was (Author: georryhuang):
I guess this is not a bug.
{code:java}
size [128 MB, 8.00 EB, 8.00 EB); {code}
represents [*minCompactSize*, *maxCompactSize*, *offPeakMaxCompactSize*).
The default value of *maxCompactSize(*hbase.hstore.compaction.max.size*)* and
*offPeakMaxCompactSize(*hbase.hstore.compaction.max.size.offpeak*)* is
*Long.MAX*, so the display is 8.00EB.
But the information here is easy to misunderstand. How about changing it like
this?
{code:java}
size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00
EB); files [minFilesToCompact:3, maxFilesToCompact:10);
{code}
> CompactionConfiguration logs unrealistic store file sizes
> -
>
> Key: HBASE-25090
> URL: https://issues.apache.org/jira/browse/HBASE-25090
> Project: HBase
> Issue Type: Bug
> Components: Compaction
>Affects Versions: 2.3.2
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Minor
>
> I noticed this in the debug logging of a unit test that makes use of a
> MiniCluster. I'm pretty sure {{8.00 EB}} -- exabytes?? -- is not an accurate
> reporting of the size of these HFiles.
> {noformat}
> 2020-09-22 16:20:57,996 INFO
> [StoreOpener-5e0b426a952230abb65623fc54869833-1]
> compactions.CompactionConfiguration(177): size [128 MB, 8.00 EB, 8.00 EB);
> files [3, 10); ...
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25090) CompactionConfiguration logs unrealistic store file sizes
[
https://issues.apache.org/jira/browse/HBASE-25090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219100#comment-17219100
]
Zhuoyue Huang commented on HBASE-25090:
---
I guess this is not a bug.
{code:java}
size [128 MB, 8.00 EB, 8.00 EB); {code}
represents [*minCompactSize*, *maxCompactSize*, *offPeakMaxCompactSize*).
The default value of *maxCompactSize(*hbase.hstore.compaction.max.size*)* and
*offPeakMaxCompactSize(*hbase.hstore.compaction.max.size.offpeak*)* is
*Long.MAX*, so the display is 8.00EB.
But the information here is easy to misunderstand. How about changing it like
this?
{code:java}
size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00
EB); files [minFilesToCompact:3, maxFilesToCompact:10);
{code}
> CompactionConfiguration logs unrealistic store file sizes
> -
>
> Key: HBASE-25090
> URL: https://issues.apache.org/jira/browse/HBASE-25090
> Project: HBase
> Issue Type: Bug
> Components: Compaction
>Affects Versions: 2.3.2
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Minor
>
> I noticed this in the debug logging of a unit test that makes use of a
> MiniCluster. I'm pretty sure {{8.00 EB}} -- exabytes?? -- is not an accurate
> reporting of the size of these HFiles.
> {noformat}
> 2020-09-22 16:20:57,996 INFO
> [StoreOpener-5e0b426a952230abb65623fc54869833-1]
> compactions.CompactionConfiguration(177): size [128 MB, 8.00 EB, 8.00 EB);
> files [3, 10); ...
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] sguggilam commented on pull request #2578: HBASE-24768 Clear cached service kerberos ticket in case of SASL fail…
sguggilam commented on pull request #2578: URL: https://github.com/apache/hbase/pull/2578#issuecomment-714555919 @apurtell @abhishek-chouhan Please review This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2579: HBASE-24999 Master manages ReplicationServers
Apache-HBase commented on pull request #2579: URL: https://github.com/apache/hbase/pull/2579#issuecomment-714553561 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-24666 Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 59s | HBASE-24666 passed | | +1 :green_heart: | compile | 2m 17s | HBASE-24666 passed | | +1 :green_heart: | shadedjars | 9m 0s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 0s | HBASE-24666 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 19s | the patch passed | | +1 :green_heart: | compile | 2m 7s | the patch passed | | +1 :green_heart: | javac | 2m 7s | the patch passed | | +1 :green_heart: | shadedjars | 8m 35s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 1s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 3s | hbase-protocol-shaded in the patch passed. | | -1 :x: | unit | 156m 14s | hbase-server in the patch failed. | | | | 194m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2579 | | Optional Tests | unit javac javadoc shadedjars compile | | uname | Linux 20c2fdb2c851 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-24666 / 1a64f1a0db | | Default Java | 1.8.0_232 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/2/testReport/ | | Max. process+thread count | 3819 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
