[GitHub] [hbase] Apache-HBase commented on pull request #3665: HBASE-26248 Should find a suitable way to let users specify the store…
Apache-HBase commented on pull request #3665: URL: https://github.com/apache/hbase/pull/3665#issuecomment-915808756 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-26067 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 26s | HBASE-26067 passed | | +1 :green_heart: | compile | 3m 22s | HBASE-26067 passed | | +1 :green_heart: | checkstyle | 1m 10s | HBASE-26067 passed | | +1 :green_heart: | spotbugs | 2m 13s | HBASE-26067 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 2s | the patch passed | | +1 :green_heart: | compile | 3m 20s | the patch passed | | +1 :green_heart: | javac | 3m 20s | the patch passed | | +1 :green_heart: | checkstyle | 1m 8s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 21m 28s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 26s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 53m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3665/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3665 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux bef4838c8413 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-26067 / 27fd631b52 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3665/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3667: HBASE-21521 Expose master startup status via web UI
Apache-HBase commented on pull request #3667: URL: https://github.com/apache/hbase/pull/3667#issuecomment-915768999 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 2s | master passed | | +1 :green_heart: | compile | 1m 41s | master passed | | +1 :green_heart: | shadedjars | 10m 37s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 8s | the patch passed | | +1 :green_heart: | compile | 1m 35s | the patch passed | | +1 :green_heart: | javac | 1m 35s | the patch passed | | +1 :green_heart: | shadedjars | 10m 28s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 53s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 246m 44s | hbase-server in the patch failed. | | | | 288m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3667 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 8d022c25879d 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b8ba3f788b | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/2/testReport/ | | Max. process+thread count | 2923 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3667: HBASE-21521 Expose master startup status via web UI
Apache-HBase commented on pull request #3667: URL: https://github.com/apache/hbase/pull/3667#issuecomment-915753351 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 1s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 31s | master passed | | +1 :green_heart: | compile | 1m 3s | master passed | | +1 :green_heart: | shadedjars | 9m 2s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 5s | the patch passed | | +1 :green_heart: | compile | 1m 5s | the patch passed | | +1 :green_heart: | javac | 1m 5s | the patch passed | | +1 :green_heart: | shadedjars | 9m 5s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 212m 47s | hbase-server in the patch failed. | | | | 245m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3667 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ee0d885c2304 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b8ba3f788b | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/2/testReport/ | | Max. process+thread count | 3426 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-26243) Fix typo for file 'hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java'
[
https://issues.apache.org/jira/browse/HBASE-26243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang resolved HBASE-26243.
---
Fix Version/s: 2.4.7
2.3.7
3.0.0-alpha-2
2.5.0
Hadoop Flags: Reviewed
Resolution: Fixed
Pushed to branch-2.3+.
Thanks [~hapihu] for contributing.
> Fix typo for file
> 'hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java'
> --
>
> Key: HBASE-26243
> URL: https://issues.apache.org/jira/browse/HBASE-26243
> Project: HBase
> Issue Type: Improvement
>Reporter: wuguihu
>Assignee: wuguihu
>Priority: Trivial
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.3.7, 2.4.7
>
> Attachments: HBASE-26243.patch
>
>
> Fix typo for file
> 'hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java'
> {code:bash}
> hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java:1823:
> mulitple ==> multiple
> hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java:3562:
> compatiblity ==> compatibility
> hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java:3864:
> supportted ==> supported
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache9 merged pull request #3650: [HBASE-26243]Fix typo for file "hbase-server/src/main/java/org/apache…
Apache9 merged pull request #3650: URL: https://github.com/apache/hbase/pull/3650 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 merged pull request #3656: HBASE-26224 Introduce a MigrationStoreFileTracker to support migratin…
Apache9 merged pull request #3656: URL: https://github.com/apache/hbase/pull/3656 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26263) [Rolling Upgrading] Persist the StoreFileTracker configurations to TableDescriptor for existing tables
[ https://issues.apache.org/jira/browse/HBASE-26263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26263: -- Summary: [Rolling Upgrading] Persist the StoreFileTracker configurations to TableDescriptor for existing tables (was: [Rolling Upgrading] Persist the StoreEngine configurations to TableDescriptor for existing tables) > [Rolling Upgrading] Persist the StoreFileTracker configurations to > TableDescriptor for existing tables > -- > > Key: HBASE-26263 > URL: https://issues.apache.org/jira/browse/HBASE-26263 > Project: HBase > Issue Type: Sub-task > Components: HFile, master >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-26246) Persist the StoreFileTracker configurations to TableDescriptor when creating table
[ https://issues.apache.org/jira/browse/HBASE-26246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26246: -- Summary: Persist the StoreFileTracker configurations to TableDescriptor when creating table (was: Persist the store engine configuration to TableDescriptor when creating table) > Persist the StoreFileTracker configurations to TableDescriptor when creating > table > -- > > Key: HBASE-26246 > URL: https://issues.apache.org/jira/browse/HBASE-26246 > Project: HBase > Issue Type: Sub-task > Components: HFile, master >Reporter: Duo Zhang >Assignee: Wellington Chevreuil >Priority: Major > > As discussed in this section in the design doc: > https://docs.google.com/document/d/16Nr1Fn3VaXuz1g1FTiME-bnGR3qVK5B-raXshOkDLcY/edit#heading=h.78r2mdeyquug > If we use different SFT implementation at master side and region server side, > it is likely to cause data loss, which is a very serious misconfiguration > problem. > A possible solution is to make sure that master and region server always load > the configurations about StoreEngine from the same place. To archive this, a > possible way is to always set the StoreEngine configurations to the > TableDescriptor, even if user does not explicitly set it when creating a > table. > And also, when upgrading, we should check whether the existing tables have > StoreEngine configurations, if not, we need to set them. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26245) Store region server list in master local region
[
https://issues.apache.org/jira/browse/HBASE-26245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412271#comment-17412271
]
Duo Zhang commented on HBASE-26245:
---
{quote}
At a minimum, if we track RegionServers which are "dead but not yet processed"
in the Master Region, I think that handles our biggest "we don't know if it's
safe to schedule SCP" concern.
{quote}
Practically, we do not have a list like this. If we want to process a dead
region server, we will schedule a SCP, and it will be recorded in the procedure
store. There is only a very short interval here, I do not think it worth to add
one more step to store the region server to other places...
The root problem here, is we need to get the old live region server list, and
then compare it with the current live region server list, to find out the dead
region servers need to process.
In general, the region server list could also be constructed by scanning
meta(this should the typical bigtable way), but there is a cyclic dependency
that, we need SCP to bring meta online first. I think there could be tricky way
to break the tie. As we can load the location of meta region first, then we
could compare it with the current live region servers, if it is dead, then we
could schedule a SCP for the region server first to bring meta online, and then
we could scan meta to find out other region servers.
But there is still a problem for replication. In the current implementation, we
rely on SCP to assign the replication queues for a dead region server to other
region servers, so even if the region server does not carry any regions, we
still need to schedule a SCP for it, as maybe it held some regions in the past
and then all of them were moved elsewhere, but it still has some unreplicated
wals...
So, this is a complicated problem... We need to discuss more here.
Thanks [~elserj] and [~zyork] for chimming in.
> Store region server list in master local region
> ---
>
> Key: HBASE-26245
> URL: https://issues.apache.org/jira/browse/HBASE-26245
> Project: HBase
> Issue Type: Brainstorming
> Components: master, Zookeeper
>Reporter: Duo Zhang
>Priority: Major
>
> Just a simple idea still need to be polished.
> For large clusters, ZooKeeper could be a bottle neck, there are some related
> issues to not track region server list as much as possible, but what if we
> want to do more, just do not register region server on zk?
> I think, zookeeper here is act as something like a service registry, we need
> the list to get all the region servers, and also need to know the changes of
> the list.
> But in fact, we could also kill region server from master side, so the latter
> one could be done by a periodical heartbeat check daemon on master. And for
> the former one, we could store the list in master local region, so when
> master restart, it could setup the region server list by load from master
> local region.
> And there are mainly two other side effect which are good:
> 1. We do not need to list the WAL directory on HDFS to find the previous
> region servers, for scheduling SCP. This could make it possible to restart a
> new HBase cluster based on only the root directory.
> 2. For now, region server needs to register to HMaster first, and then put
> its node on ZooKeeper, if it fails between these two actions, there is no way
> for HMaster to clean this dead server, as it never expires on ZooKeeper.
> There should be a related issue. If we just do not store a node on zk, then
> this problem is also gone.
> Of course, there will still be lots of other problems, like whether we need
> another heartbeat call as reportForDuty is a bit heavy as we also report the
> region list, etc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26258) Universal compression support via aircompressor
[ https://issues.apache.org/jira/browse/HBASE-26258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412269#comment-17412269 ] Duo Zhang commented on HBASE-26258: --- OK, got it. Fallback is suitable here. Thanks for clarifying this. > Universal compression support via aircompressor > --- > > Key: HBASE-26258 > URL: https://issues.apache.org/jira/browse/HBASE-26258 > Project: HBase > Issue Type: Improvement > Components: HFile, Operability >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-2 > > > Some Hadoop compression codecs became more available in recent Hadoop 3.x > releases, addressed by HBASE-25940. This is nice but still requires native > platform support, which to state the obvious is not available on all > platforms and architectures, even if native libaries for some are bundled > into jars. > Airlift's aircompressor > (https://search.maven.org/artifact/io.airlift/aircompressor) is an Apache 2 > licensed library, for Java 8 and up, available in Maven central, which > provides pure Java implementations of desirable compression algorithms gzip, > lz4, lzo, snappy, and zstd, and Hadoop compression codecs for same, claiming > "_they are typically 300% faster than the JNI wrappers_." > (https://github.com/airlift/aircompressor). This library is under active > development and has up to date releases because it is used by Trino. > We have another project that depends on universal availability of SNAPPY. I > would like to make this change as a general improvement which also satisfies > that requirement. (The as yet unnamed project will be contributed later.) It > will be a very nice-to-have to have universal ZSTD support available as well. > Proposed changes: > * Modify Compression.java such that compression codec implementation classes > can be specified by configuration. Currently they are hardcoded as strings. > * Pull in aircompressor as a 'compile' time dependency so it will be bundled > into our build and made available on the server classpath. > * Modify Compression.java to fall back to an aircompressor pure Java > implementation if schema specifies a compression algorithm, a Hadoop native > codec was specified as desired implementation, but the requisite native > support is somehow not available. > The combination of these changes will provide universal (pure Java) support > for these desired and desirable compression codecs while retaining default > behavior, which is to load and utilize Hadoop native implementations of same, > if native support is available. They will also let you override this default > if you wish to chase the claimed benefits of the pure Java alternatives. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 commented on pull request #3666: HBASE-26246 Persist the store engine configuration to TableDescriptor when creating table
Apache9 commented on pull request #3666: URL: https://github.com/apache/hbase/pull/3666#issuecomment-915694391 > Actually, now got bit confused by "persisting store engine configuration". I thought StoreFileTracker impl config would be interchangeable with StoreEngine, and we really don't want to mess with different StoreFileTracker impl being used by master and RSes operations, so we define table of CF descriptors as a global point. > > Are you thinking we should actually put the StoreEngine impl config here, and then, each StoreEngine impl would be responsible to configure its StoreFileTracker impl? Yes, I was thinking we should store the StoreEngine config as a whole. For example, if we use StripeCompaction in the past but later we change the default config to DateTieredCompaction, should we switch all the tables' config? At least for me, I do not think so. But anyway, changing stripe to date tiered will not cause data loss, only impact performance, so could be a separated issue, and let;s discuss more. Let's change the title to narrow down the scope to store file tracker only. And I think we'd beter add a method in StoreFileTracker, like `void persistConfiguration(TableDescriptorBuilder builder);` To store all the necessary configurations to the TableDescriptor. For example, for MigrationStoreFileTracker, we need to store the src and dst store file tracker implementation as well. WDYT? @wchevreuil Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3667: HBASE-21521 Expose master startup status via web UI
Apache-HBase commented on pull request #3667: URL: https://github.com/apache/hbase/pull/3667#issuecomment-915681320 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 39s | master passed | | +1 :green_heart: | compile | 3m 10s | master passed | | +1 :green_heart: | checkstyle | 1m 4s | master passed | | +1 :green_heart: | spotbugs | 2m 4s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 41s | the patch passed | | +1 :green_heart: | compile | 3m 8s | the patch passed | | +1 :green_heart: | javac | 3m 8s | the patch passed | | -0 :warning: | checkstyle | 1m 2s | hbase-server: The patch generated 4 new + 91 unchanged - 0 fixed = 95 total (was 91) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 16s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 15s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 46m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3667 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux a9e1329bcc06 4.15.0-151-generic #157-Ubuntu SMP Fri Jul 9 23:07:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b8ba3f788b | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 95 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26269) Shell 'load' command behavior has changed
[
https://issues.apache.org/jira/browse/HBASE-26269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412252#comment-17412252
]
Sean Busbey commented on HBASE-26269:
-
This smells like a consequence of HBASE-14067
> Shell 'load' command behavior has changed
> -
>
> Key: HBASE-26269
> URL: https://issues.apache.org/jira/browse/HBASE-26269
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.5
>Reporter: Nick Dimiduk
>Priority: Major
>
> After upgrade fro 2.3.x to 2.4.x, we noticed that the behavior of the shell's
> {{load}} command has changed.
> Given a file of commands, i.e.
> {noformat}
> $ echo 'list' > /tmp/file
> {noformat}
> 2.3.x:
> {noformat}
> $ hbase shell
> hbase(main):001:0> load '/tmp/file'
> TABLE
>
>
> TestTable
>
>
> 1 row(s)
> Took 0.3076 seconds
>
>
> => true
> hbase(main):002:0>
> {noformat}
> branch-2.4:
> {noformat}
> $ hbase shell
> hbase:001:0> load '/tmp/file'
> Traceback (most recent call last):
> NameError (undefined local variable or method `list' for main:Object)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3667: HBASE-21521 Expose master startup status via web UI
Apache-HBase commented on pull request #3667: URL: https://github.com/apache/hbase/pull/3667#issuecomment-915660911 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 40s | master passed | | +1 :green_heart: | compile | 1m 6s | master passed | | +1 :green_heart: | shadedjars | 9m 46s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 11s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 10s | the patch passed | | +1 :green_heart: | compile | 1m 8s | the patch passed | | +1 :green_heart: | javac | 1m 8s | the patch passed | | +1 :green_heart: | shadedjars | 9m 10s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 213m 6s | hbase-server in the patch failed. | | | | 248m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3667 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 70f7766262bc 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b8ba3f788b | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/1/testReport/ | | Max. process+thread count | 3343 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3667: HBASE-21521 Expose master startup status via web UI
Apache-HBase commented on pull request #3667: URL: https://github.com/apache/hbase/pull/3667#issuecomment-915660858 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 19s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 9s | master passed | | +1 :green_heart: | compile | 1m 19s | master passed | | +1 :green_heart: | shadedjars | 9m 10s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 49s | the patch passed | | +1 :green_heart: | compile | 1m 20s | the patch passed | | +1 :green_heart: | javac | 1m 20s | the patch passed | | +1 :green_heart: | shadedjars | 9m 9s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 211m 42s | hbase-server in the patch failed. | | | | 247m 31s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3667 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 39a8717ec37b 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b8ba3f788b | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/1/testReport/ | | Max. process+thread count | 2993 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26269) Shell 'load' command behavior has changed
[
https://issues.apache.org/jira/browse/HBASE-26269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-26269:
-
Summary: Shell 'load' command behavior has changed (was: Shell no longer
supports 'load' command)
> Shell 'load' command behavior has changed
> -
>
> Key: HBASE-26269
> URL: https://issues.apache.org/jira/browse/HBASE-26269
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.5
>Reporter: Nick Dimiduk
>Priority: Major
>
> After upgrade fro 2.3.x to 2.4.x, we noticed that the behavior of the shell's
> {{load}} command has changed.
> Given a file of commands, i.e.
> {noformat}
> $ echo 'list' > /tmp/file
> {noformat}
> 2.3.x:
> {noformat}
> $ hbase shell
> hbase(main):001:0> load '/tmp/file'
> TABLE
>
>
> TestTable
>
>
> 1 row(s)
> Took 0.3076 seconds
>
>
> => true
> hbase(main):002:0>
> {noformat}
> branch-2.4:
> {noformat}
> $ hbase shell
> hbase:001:0> load '/tmp/file'
> Traceback (most recent call last):
> NameError (undefined local variable or method `list' for main:Object)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-26269) Shell no longer supports 'load' command
Nick Dimiduk created HBASE-26269:
Summary: Shell no longer supports 'load' command
Key: HBASE-26269
URL: https://issues.apache.org/jira/browse/HBASE-26269
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 2.4.5, 3.0.0-alpha-2
Reporter: Nick Dimiduk
After upgrade fro 2.3.x to 2.4.x, we noticed that the behavior of the shell's
{{load}} command has changed.
Given a file of commands, i.e.
{noformat}
$ echo 'list' > /tmp/file
{noformat}
2.3.x:
{noformat}
$ hbase shell
hbase(main):001:0> load '/tmp/file'
TABLE
TestTable
1 row(s)
Took 0.3076 seconds
=> true
hbase(main):002:0>
{noformat}
branch-2.4:
{noformat}
$ hbase shell
hbase:001:0> load '/tmp/file'
Traceback (most recent call last):
NameError (undefined local variable or method `list' for main:Object)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-26023) tableSkewCostFunction aggregate cost per table incorrectly
[
https://issues.apache.org/jira/browse/HBASE-26023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Clara Xiong updated HBASE-26023:
Summary: tableSkewCostFunction aggregate cost per table incorrectly (was:
Overhaul of test cluster set up for table skew)
> tableSkewCostFunction aggregate cost per table incorrectly
> --
>
> Key: HBASE-26023
> URL: https://issues.apache.org/jira/browse/HBASE-26023
> Project: HBase
> Issue Type: Sub-task
> Components: Balancer, test
>Reporter: Clara Xiong
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.6, 2.4.5
>
>
> There is another bug in the original tableSkew cost function for aggregation
> of the cost per table:
> If we have 10 regions, one per table, evenly distributed on 10 nodes, the
> cost is scale to 1.0.
> The more tables we have, the closer the value will be to 1.0. The cost
> function becomes useless.
> All the balancer tests were set up with large numbers of tables with minimal
> regions per table. This artificially inflates the total cost and trigger
> balancer runs. With this fix on TableSkewFunction, we need to overhaul the
> tests too. We also need to add tests that reflect more diversified scenarios
> for table distribution such as large tables with large numbers of regions.
> {code:java}
> protected double cost() {
> double max = cluster.numRegions;
> double min = ((double) cluster.numRegions) / cluster.numServers;
> double value = 0;
> for (int i = 0; i < cluster.numMaxRegionsPerTable.length; i++) {
> value += cluster.numMaxRegionsPerTable[i];
> }
> LOG.info("min = {}, max = {}, cost= {}", min, max, value);
> return scale(min, max, value);
> }
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25769) Update default weight of cost functions
[ https://issues.apache.org/jira/browse/HBASE-25769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412231#comment-17412231 ] Clara Xiong commented on HBASE-25769: - [~Xiaolin Ha] More updates for byTable option. We have seen it enabled on a different large cluster ( close to 1000 RS )which worked poorly too. By table works fine with many small tables and with at most a large table on a large cluster. The fix in https://issues.apache.org/jira/browse/HBASE-25739 make tableSkewCostFunction finally work and we switched to rely on it by increasing its weight other than by table option. > Update default weight of cost functions > --- > > Key: HBASE-25769 > URL: https://issues.apache.org/jira/browse/HBASE-25769 > Project: HBase > Issue Type: Sub-task > Components: Balancer >Reporter: Clara Xiong >Priority: Major > > In production, we have seen some critical big tables that handle majority of > the load. Table Skew is becoming more important. With the update of table > skew function, balancer finally works for large table distribution on large > cluster. We should increase the weight from 35 to a level comparable to > region count skew: 500. We can even push further to replace region count skew > by table skew since the latter works in the same way and account for region > distribution per node. > Another weight we found helpful to increase is for store file size cost > function. Ideally if normalizer works perfectly, we don't need to worry about > it since region count skew would have accounted for it. But we are often in a > situation it doesn't. Store file distribution needs to be given more way as > accommodation. we tested changing it from 5 to 200 and it works fine. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25625) StochasticBalancer CostFunctions needs a better way to evaluate resource distribution
[ https://issues.apache.org/jira/browse/HBASE-25625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412227#comment-17412227 ] Clara Xiong commented on HBASE-25625: - With the release of https://issues.apache.org/jira/browse/HBASE-25739, most of the problem we observed are gone. We will revisit this Jira and the PR as needed. > StochasticBalancer CostFunctions needs a better way to evaluate resource > distribution > - > > Key: HBASE-25625 > URL: https://issues.apache.org/jira/browse/HBASE-25625 > Project: HBase > Issue Type: Improvement > Components: Balancer, master >Reporter: Clara Xiong >Assignee: Clara Xiong >Priority: Major > > Currently CostFunctions including RegionCountSkewCostFunctions, > PrimaryRegionCountSkewCostFunctions and all load cost functions calculate the > unevenness of the distribution by getting the sum of deviation per region > server. This simple implementation works when the cluster is small. But when > the cluster get larger with more region servers and regions, it doesn't work > well with hot spots or a small number of unbalanced servers. The proposal is > to use the standard deviation of the count per region server to capture the > existence of a small portion of region servers with overwhelming > load/allocation. > TableSkewCostFunction uses the sum of the max deviation region per server for > all tables as the measure of unevenness. It doesn't work in a very common > scenario in operations. Say we have 100 regions on 50 nodes, two on each. We > add 50 new nodes and they have 0 each. The max deviation from the mean is 1, > compared to 99 in the worst case scenario of 100 regions on a single server. > The normalized cost is 1/99 = 0.011 < default threshold of 0.05. Balancer > wouldn't move. The proposal is to use the standard deviation of the count > per region server to detect this scenario, generating a cost of 3.1/31 = 0.1 > in this case. > Patch is in test and will follow shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-26023) Overhaul of test cluster set up for table skew
[
https://issues.apache.org/jira/browse/HBASE-26023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Clara Xiong resolved HBASE-26023.
-
Fix Version/s: 3.0.0-alpha-1
2.3.6
2.4.5
Resolution: Fixed
This was fixed in the same PRs for
https://issues.apache.org/jira/browse/HBASE-25739
> Overhaul of test cluster set up for table skew
> --
>
> Key: HBASE-26023
> URL: https://issues.apache.org/jira/browse/HBASE-26023
> Project: HBase
> Issue Type: Sub-task
> Components: Balancer, test
>Reporter: Clara Xiong
>Priority: Major
> Fix For: 2.4.5, 2.3.6, 3.0.0-alpha-1
>
>
> There is another bug in the original tableSkew cost function for aggregation
> of the cost per table:
> If we have 10 regions, one per table, evenly distributed on 10 nodes, the
> cost is scale to 1.0.
> The more tables we have, the closer the value will be to 1.0. The cost
> function becomes useless.
> All the balancer tests were set up with large numbers of tables with minimal
> regions per table. This artificially inflates the total cost and trigger
> balancer runs. With this fix on TableSkewFunction, we need to overhaul the
> tests too. We also need to add tests that reflect more diversified scenarios
> for table distribution such as large tables with large numbers of regions.
> {code:java}
> protected double cost() {
> double max = cluster.numRegions;
> double min = ((double) cluster.numRegions) / cluster.numServers;
> double value = 0;
> for (int i = 0; i < cluster.numMaxRegionsPerTable.length; i++) {
> value += cluster.numMaxRegionsPerTable[i];
> }
> LOG.info("min = {}, max = {}, cost= {}", min, max, value);
> return scale(min, max, value);
> }
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-26237) Improve computation complexity for primaryRegionCountSkewCostFunctio
[ https://issues.apache.org/jira/browse/HBASE-26237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Clara Xiong resolved HBASE-26237. - Fix Version/s: 3.0.0-alpha-2 Resolution: Fixed > Improve computation complexity for primaryRegionCountSkewCostFunctio > > > Key: HBASE-26237 > URL: https://issues.apache.org/jira/browse/HBASE-26237 > Project: HBase > Issue Type: Sub-task > Components: Balancer >Reporter: Clara Xiong >Priority: Minor > Fix For: 3.0.0-alpha-2 > > > Recomputation of primaryRegionCountSkewCostFunction can be reduced from O(n ) > to O(1) by only incrementing the destination and decrementing the source > instead of full recompute. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26252) Add support for reloading balancer configs with BalanceRequest
[
https://issues.apache.org/jira/browse/HBASE-26252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412209#comment-17412209
]
Bryan Beaudreault commented on HBASE-26252:
---
Running the dry run with temporary config overrides would be handled by a
separate task. This task is more about making it easier to deploy changes for
real, once you’ve decided to deploy them, by streamlining the dynamic reload.
> Add support for reloading balancer configs with BalanceRequest
> --
>
> Key: HBASE-26252
> URL: https://issues.apache.org/jira/browse/HBASE-26252
> Project: HBase
> Issue Type: Sub-task
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Minor
>
> It's currently a pain to iterate on balancer configs. You need to make
> changes in hbase-site.xml, then find the full ServerName for the active
> HMaster, then execute {{update_configuration ''}}in the
> shell, then run the balancer.
> Finding the ServerName is actually quite annoying. The best way I've found is
> to look at the JMX dump and find {{tag.serverName}}, but that takes a bunch
> of steps.
> We can make this a good deal more convenient by adding direct support for
> reloading the balancer configs into the {{balance}} command.
> This could look something like:
> {{shell> balance \{RELOAD_CONFIGS => true}}}
> Alternatively, we could add another string arg like:
> {{shell> balance 'reload_config'}}
> Either way, we'd add a new
> {{BalanceRequest$Builder#setReloadConfig(boolean)}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26252) Add support for reloading balancer configs with BalanceRequest
[
https://issues.apache.org/jira/browse/HBASE-26252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412208#comment-17412208
]
Bryan Beaudreault commented on HBASE-26252:
---
Thanks for the clarification [~claraxiong]. Let me better explain the workflow
as it exists today and with this change:
*Today*
If you want to make a change to the balancer, you must:
* update hbase-site.xml
* restart your HMaster OR run {{update_configuration '}}
in hbase shell
** If you decide to use {{update_configuration}}, you must figure out the
ServerName for the active master. The only way I know to do this is to look at
tag.serverName in Metrics Dump endpoint, or look in hmaster logs. Both of these
are really annoying to do and not straightforward if you don't know how.
** You may also not even know about {{update_configuration}}
* run the balancer with {{balance}} in hbase shell, or let it run by the
default chore
I don't believe these configs are dynamically reloaded unless you run
update_configuration. Correct me if I'm wrong.
*With this change*
All you have to do at this point is:
* update your hbase-site.xml
* run balancer with {{balance 'reload_configs'}}{{}}
I think this is much more straight forward and it better exposes the fact that
these configs can be reloaded dynamically.
> Add support for reloading balancer configs with BalanceRequest
> --
>
> Key: HBASE-26252
> URL: https://issues.apache.org/jira/browse/HBASE-26252
> Project: HBase
> Issue Type: Sub-task
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Minor
>
> It's currently a pain to iterate on balancer configs. You need to make
> changes in hbase-site.xml, then find the full ServerName for the active
> HMaster, then execute {{update_configuration ''}}in the
> shell, then run the balancer.
> Finding the ServerName is actually quite annoying. The best way I've found is
> to look at the JMX dump and find {{tag.serverName}}, but that takes a bunch
> of steps.
> We can make this a good deal more convenient by adding direct support for
> reloading the balancer configs into the {{balance}} command.
> This could look something like:
> {{shell> balance \{RELOAD_CONFIGS => true}}}
> Alternatively, we could add another string arg like:
> {{shell> balance 'reload_config'}}
> Either way, we'd add a new
> {{BalanceRequest$Builder#setReloadConfig(boolean)}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-26268) AccessController is missing a couple AdminService methods
Bryan Beaudreault created HBASE-26268: - Summary: AccessController is missing a couple AdminService methods Key: HBASE-26268 URL: https://issues.apache.org/jira/browse/HBASE-26268 Project: HBase Issue Type: Bug Reporter: Bryan Beaudreault Sorry for the vague title, not sure if it'd be better to create individual jiras. I discovered that the following two AdminService endpoints are not covered by AccessController: * updateConfiguration * clearRegionBlockCache There may be others, I haven't done a full audit. We should add coprocessor hooks for these and wrap with AccessController, especially since they can both have an affect on the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3669: HBASE-26255 Add an option to use region location from meta table in T…
Apache-HBase commented on pull request #3669: URL: https://github.com/apache/hbase/pull/3669#issuecomment-915599072 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 44s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.4 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 38s | branch-2.4 passed | | +1 :green_heart: | compile | 0m 33s | branch-2.4 passed | | +1 :green_heart: | shadedjars | 8m 13s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | branch-2.4 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 22s | the patch passed | | +1 :green_heart: | compile | 0m 32s | the patch passed | | +1 :green_heart: | javac | 0m 32s | the patch passed | | +1 :green_heart: | shadedjars | 8m 41s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 17m 5s | hbase-mapreduce in the patch passed. | | | | 50m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3669/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3669 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7a6195e378b5 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.4 / b78d35598c | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3669/1/testReport/ | | Max. process+thread count | 3498 (vs. ulimit of 12500) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3669/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3670: HBASE-26255 Add an option to use region location from meta table in T…
Apache-HBase commented on pull request #3670: URL: https://github.com/apache/hbase/pull/3670#issuecomment-915594898 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 36s | branch-2.3 passed | | +1 :green_heart: | compile | 0m 52s | branch-2.3 passed | | +1 :green_heart: | checkstyle | 0m 22s | branch-2.3 passed | | +1 :green_heart: | spotbugs | 0m 47s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 11s | the patch passed | | +1 :green_heart: | compile | 0m 49s | the patch passed | | -0 :warning: | javac | 0m 49s | hbase-mapreduce generated 1 new + 197 unchanged - 1 fixed = 198 total (was 198) | | -0 :warning: | checkstyle | 0m 19s | hbase-mapreduce: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 0s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 0m 55s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 36m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3670/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3670 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux bed4c8b98bec 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 10373c37b0 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3670/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-mapreduce.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3670/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-mapreduce.txt | | Max. process+thread count | 96 (vs. ulimit of 12500) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3670/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3670: HBASE-26255 Add an option to use region location from meta table in T…
Apache-HBase commented on pull request #3670: URL: https://github.com/apache/hbase/pull/3670#issuecomment-915593987 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 17s | branch-2.3 passed | | +1 :green_heart: | compile | 0m 29s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 5m 52s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 52s | the patch passed | | +1 :green_heart: | compile | 0m 29s | the patch passed | | +1 :green_heart: | javac | 0m 29s | the patch passed | | +1 :green_heart: | shadedjars | 5m 51s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 11m 29s | hbase-mapreduce in the patch passed. | | | | 35m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3670/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3670 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 90d7ef6c8a6a 4.15.0-151-generic #157-Ubuntu SMP Fri Jul 9 23:07:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 10373c37b0 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3670/1/testReport/ | | Max. process+thread count | 4062 (vs. ulimit of 12500) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3670/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3670: HBASE-26255 Add an option to use region location from meta table in T…
Apache-HBase commented on pull request #3670: URL: https://github.com/apache/hbase/pull/3670#issuecomment-915593486 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | | -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 35s | branch-2.3 passed | | +1 :green_heart: | compile | 0m 27s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 5m 10s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 14s | the patch passed | | +1 :green_heart: | compile | 0m 26s | the patch passed | | +1 :green_heart: | javac | 0m 26s | the patch passed | | +1 :green_heart: | shadedjars | 6m 14s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 20s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 12m 20s | hbase-mapreduce in the patch failed. | | | | 34m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3670/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3670 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux cf47dc1f47e7 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 10373c37b0 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3670/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-mapreduce.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3670/1/testReport/ | | Max. process+thread count | 3940 (vs. ulimit of 12500) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3670/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3669: HBASE-26255 Add an option to use region location from meta table in T…
Apache-HBase commented on pull request #3669: URL: https://github.com/apache/hbase/pull/3669#issuecomment-915591961 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.4 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 48s | branch-2.4 passed | | +1 :green_heart: | compile | 0m 52s | branch-2.4 passed | | +1 :green_heart: | checkstyle | 0m 21s | branch-2.4 passed | | +1 :green_heart: | spotbugs | 0m 46s | branch-2.4 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 18s | the patch passed | | +1 :green_heart: | compile | 0m 47s | the patch passed | | -0 :warning: | javac | 0m 47s | hbase-mapreduce generated 1 new + 197 unchanged - 1 fixed = 198 total (was 198) | | -0 :warning: | checkstyle | 0m 19s | hbase-mapreduce: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 16s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 0m 54s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 36m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3669/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3669 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux ed742b5d1370 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.4 / b78d35598c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3669/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-mapreduce.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3669/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-mapreduce.txt | | Max. process+thread count | 96 (vs. ulimit of 12500) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3669/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3669: HBASE-26255 Add an option to use region location from meta table in T…
Apache-HBase commented on pull request #3669: URL: https://github.com/apache/hbase/pull/3669#issuecomment-915590919 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.4 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 43s | branch-2.4 passed | | +1 :green_heart: | compile | 0m 23s | branch-2.4 passed | | +1 :green_heart: | shadedjars | 6m 3s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | branch-2.4 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | compile | 0m 26s | the patch passed | | +1 :green_heart: | javac | 0m 26s | the patch passed | | +1 :green_heart: | shadedjars | 6m 0s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 19s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 12m 23s | hbase-mapreduce in the patch failed. | | | | 35m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3669/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3669 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ee8f4bf52d98 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.4 / b78d35598c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3669/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-mapreduce.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3669/1/testReport/ | | Max. process+thread count | 4276 (vs. ulimit of 12500) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3669/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26258) Universal compression support via aircompressor
[ https://issues.apache.org/jira/browse/HBASE-26258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-26258: Summary: Universal compression support via aircompressor (was: Universal gzip, lz4, lzo, snappy, and zstd compression support via aircompressor) > Universal compression support via aircompressor > --- > > Key: HBASE-26258 > URL: https://issues.apache.org/jira/browse/HBASE-26258 > Project: HBase > Issue Type: Improvement > Components: HFile, Operability >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-2 > > > Some Hadoop compression codecs became more available in recent Hadoop 3.x > releases, addressed by HBASE-25940. This is nice but still requires native > platform support, which to state the obvious is not available on all > platforms and architectures, even if native libaries for some are bundled > into jars. > Airlift's aircompressor > (https://search.maven.org/artifact/io.airlift/aircompressor) is an Apache 2 > licensed library, for Java 8 and up, available in Maven central, which > provides pure Java implementations of desirable compression algorithms gzip, > lz4, lzo, snappy, and zstd, and Hadoop compression codecs for same, claiming > "_they are typically 300% faster than the JNI wrappers_." > (https://github.com/airlift/aircompressor). This library is under active > development and has up to date releases because it is used by Trino. > We have another project that depends on universal availability of SNAPPY. I > would like to make this change as a general improvement which also satisfies > that requirement. (The as yet unnamed project will be contributed later.) It > will be a very nice-to-have to have universal ZSTD support available as well. > Proposed changes: > * Modify Compression.java such that compression codec implementation classes > can be specified by configuration. Currently they are hardcoded as strings. > * Pull in aircompressor as a 'compile' time dependency so it will be bundled > into our build and made available on the server classpath. > * Modify Compression.java to fall back to an aircompressor pure Java > implementation if schema specifies a compression algorithm, a Hadoop native > codec was specified as desired implementation, but the requisite native > support is somehow not available. > The combination of these changes will provide universal (pure Java) support > for these desired and desirable compression codecs while retaining default > behavior, which is to load and utilize Hadoop native implementations of same, > if native support is available. They will also let you override this default > if you wish to chase the claimed benefits of the pure Java alternatives. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3668: HBASE-26255 Add an option to use region location from meta table in T…
Apache-HBase commented on pull request #3668: URL: https://github.com/apache/hbase/pull/3668#issuecomment-915585186 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 25s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 9m 49s | branch-2 passed | | +1 :green_heart: | compile | 0m 35s | branch-2 passed | | +1 :green_heart: | shadedjars | 8m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 39s | the patch passed | | +1 :green_heart: | compile | 0m 32s | the patch passed | | +1 :green_heart: | javac | 0m 32s | the patch passed | | +1 :green_heart: | shadedjars | 8m 9s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 16m 49s | hbase-mapreduce in the patch passed. | | | | 52m 41s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3668/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3668 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 209b888ba03b 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 62c58fc11a | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3668/1/testReport/ | | Max. process+thread count | 2666 (vs. ulimit of 12500) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3668/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26267) Master initialization fails if Master Region WAL dir is missing
[
https://issues.apache.org/jira/browse/HBASE-26267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412187#comment-17412187
]
Zach York commented on HBASE-26267:
---
I have a fix for this once HBASE-24833 is in (it depends on the test Stephen
wrote)
> Master initialization fails if Master Region WAL dir is missing
> ---
>
> Key: HBASE-26267
> URL: https://issues.apache.org/jira/browse/HBASE-26267
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 2.4.6
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.4.7
>
>
> From a recent branch-2.4 build:
> {noformat}
> 2021-09-07 19:31:19,666 ERROR [master/localhost:16000:becomeActiveMaster]
> master.HMaster(159): * ABORTING master localhost,16000,1631057476442:
> Unhandled exception. Starting shutdown. *
> java.io.FileNotFoundException: File
> hdfs://localhost:8020/hbase-2.4-wals/MasterData/WALs does not exist.
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:1059)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.access$1000(DistributedFileSystem.java:131)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$24.doCall(DistributedFileSystem.java:1119)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$24.doCall(DistributedFileSystem.java:1116)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:1126)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.open(MasterRegion.java:226)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:303)
> at
> org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:839)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2189)
> at
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:512)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> If the WAL directory is missing but the Master Region already exists, we will
> try to list the contents of the Master Region's WAL directory which may or
> may not exist. If we simply check to make sure the directory exists and then
> the rest of the initialization code works as expected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3668: HBASE-26255 Add an option to use region location from meta table in T…
Apache-HBase commented on pull request #3668: URL: https://github.com/apache/hbase/pull/3668#issuecomment-915574122 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 49s | branch-2 passed | | +1 :green_heart: | compile | 0m 27s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 26s | the patch passed | | +1 :green_heart: | compile | 0m 27s | the patch passed | | +1 :green_heart: | javac | 0m 27s | the patch passed | | +1 :green_heart: | shadedjars | 6m 32s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 19s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 12m 15s | hbase-mapreduce in the patch failed. | | | | 36m 39s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3668/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3668 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 852e77ce3a06 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 62c58fc11a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3668/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-mapreduce.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3668/1/testReport/ | | Max. process+thread count | 4430 (vs. ulimit of 12500) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3668/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun opened a new pull request #3670: HBASE-26255 Add an option to use region location from meta table in T…
huaxiangsun opened a new pull request #3670: URL: https://github.com/apache/hbase/pull/3670 …ableSnapshotInputFormat (#3661) Signed-off-by: Anoop Sam John -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3668: HBASE-26255 Add an option to use region location from meta table in T…
Apache-HBase commented on pull request #3668: URL: https://github.com/apache/hbase/pull/3668#issuecomment-915570983 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 0s | branch-2 passed | | +1 :green_heart: | compile | 0m 54s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 21s | branch-2 passed | | +1 :green_heart: | spotbugs | 0m 49s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 27s | the patch passed | | +1 :green_heart: | compile | 0m 53s | the patch passed | | -0 :warning: | javac | 0m 53s | hbase-mapreduce generated 1 new + 197 unchanged - 1 fixed = 198 total (was 198) | | -0 :warning: | checkstyle | 0m 20s | hbase-mapreduce: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 25s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 0m 56s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 32m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3668/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3668 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux a1d259953223 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 62c58fc11a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3668/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-mapreduce.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3668/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-mapreduce.txt | | Max. process+thread count | 96 (vs. ulimit of 12500) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3668/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] wchevreuil commented on pull request #3666: HBASE-26246 Persist the store engine configuration to TableDescriptor when creating table
wchevreuil commented on pull request #3666: URL: https://github.com/apache/hbase/pull/3666#issuecomment-915569191 Actually, now got bit confused by "persisting store engine configuration". I thought StoreFileTracker impl config would be interchangeable with StoreEngine, and we really don't want to mess with different StoreFileTracker impl being used by master and RSes operations, so we define table of CF descriptors as a global point. Are you thinking we should actually put the StoreEngine impl config here, and then, each StoreEngine impl would be responsible to configure its StoreFileTracker impl? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun opened a new pull request #3669: HBASE-26255 Add an option to use region location from meta table in T…
huaxiangsun opened a new pull request #3669: URL: https://github.com/apache/hbase/pull/3669 …ableSnapshotInputFormat (#3661) Signed-off-by: Anoop Sam John -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3667: HBASE-21521 Expose master startup status via web UI
Apache-HBase commented on pull request #3667: URL: https://github.com/apache/hbase/pull/3667#issuecomment-915566014 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 2s | master passed | | +1 :green_heart: | compile | 3m 9s | master passed | | +1 :green_heart: | checkstyle | 1m 5s | master passed | | +1 :green_heart: | spotbugs | 2m 3s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 43s | the patch passed | | -1 :x: | compile | 1m 27s | hbase-server in the patch failed. | | -0 :warning: | javac | 1m 27s | hbase-server in the patch failed. | | -0 :warning: | checkstyle | 1m 6s | hbase-server: The patch generated 6 new + 91 unchanged - 0 fixed = 97 total (was 91) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 31s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 15s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 46m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3667 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 209b5df3f87c 4.15.0-151-generic #157-Ubuntu SMP Fri Jul 9 23:07:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b8ba3f788b | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 95 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3667/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-26267) Master initialization fails if Master Region WAL dir is missing
Josh Elser created HBASE-26267:
--
Summary: Master initialization fails if Master Region WAL dir is
missing
Key: HBASE-26267
URL: https://issues.apache.org/jira/browse/HBASE-26267
Project: HBase
Issue Type: Improvement
Components: master
Affects Versions: 2.4.6
Reporter: Josh Elser
Assignee: Josh Elser
Fix For: 2.5.0, 3.0.0-alpha-2, 2.4.7
>From a recent branch-2.4 build:
{noformat}
2021-09-07 19:31:19,666 ERROR [master/localhost:16000:becomeActiveMaster]
master.HMaster(159): * ABORTING master localhost,16000,1631057476442:
Unhandled exception. Starting shutdown. *
java.io.FileNotFoundException: File
hdfs://localhost:8020/hbase-2.4-wals/MasterData/WALs does not exist.
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:1059)
at
org.apache.hadoop.hdfs.DistributedFileSystem.access$1000(DistributedFileSystem.java:131)
at
org.apache.hadoop.hdfs.DistributedFileSystem$24.doCall(DistributedFileSystem.java:1119)
at
org.apache.hadoop.hdfs.DistributedFileSystem$24.doCall(DistributedFileSystem.java:1116)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:1126)
at
org.apache.hadoop.hbase.master.region.MasterRegion.open(MasterRegion.java:226)
at
org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:303)
at
org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:839)
at
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2189)
at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:512)
at java.lang.Thread.run(Thread.java:748)
{noformat}
If the WAL directory is missing but the Master Region already exists, we will
try to list the contents of the Master Region's WAL directory which may or may
not exist. If we simply check to make sure the directory exists and then the
rest of the initialization code works as expected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26252) Add support for reloading balancer configs with BalanceRequest
[
https://issues.apache.org/jira/browse/HBASE-26252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412181#comment-17412181
]
Clara Xiong commented on HBASE-26252:
-
Sorry if I was not clear [~bbeaudreault] My question was whether reload is
required for dry run with changed config. As to the universal usefulness, I am
not very convinced. aren't those config dynamically reloaded?
> Add support for reloading balancer configs with BalanceRequest
> --
>
> Key: HBASE-26252
> URL: https://issues.apache.org/jira/browse/HBASE-26252
> Project: HBase
> Issue Type: Sub-task
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Minor
>
> It's currently a pain to iterate on balancer configs. You need to make
> changes in hbase-site.xml, then find the full ServerName for the active
> HMaster, then execute {{update_configuration ''}}in the
> shell, then run the balancer.
> Finding the ServerName is actually quite annoying. The best way I've found is
> to look at the JMX dump and find {{tag.serverName}}, but that takes a bunch
> of steps.
> We can make this a good deal more convenient by adding direct support for
> reloading the balancer configs into the {{balance}} command.
> This could look something like:
> {{shell> balance \{RELOAD_CONFIGS => true}}}
> Alternatively, we could add another string arg like:
> {{shell> balance 'reload_config'}}
> Either way, we'd add a new
> {{BalanceRequest$Builder#setReloadConfig(boolean)}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-26195) Data is present in replicated cluster but not present in primary cluster.
[
https://issues.apache.org/jira/browse/HBASE-26195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rushabh Shah resolved HBASE-26195.
--
Resolution: Fixed
> Data is present in replicated cluster but not present in primary cluster.
> -
>
> Key: HBASE-26195
> URL: https://issues.apache.org/jira/browse/HBASE-26195
> Project: HBase
> Issue Type: Bug
> Components: Replication, wal
>Affects Versions: 1.7.0
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Major
> Fix For: 1.8.0
>
>
> We encountered a case where we are seeing some rows (via Phoenix) in
> replicated cluster but they are not present in source/active cluster.
> Triaging further we found memstore rollback logs in few of the region servers.
> {noformat}
> 2021-07-28 14:17:59,353 DEBUG [3,queue=3,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,353 DEBUG [,queue=25,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,354 DEBUG [3,queue=3,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,354 DEBUG [,queue=25,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,354 DEBUG [3,queue=3,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,354 DEBUG [,queue=25,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,355 DEBUG [3,queue=3,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,355 DEBUG [,queue=25,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,356 DEBUG [,queue=25,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> {noformat}
> Looking more into logs, found that there were some hdfs layer issues sync'ing
> wal to hdfs.
> It was taking around 6 mins to sync wal. Logs below
> {noformat}
> 2021-07-28 14:19:30,511 WARN [sync.0] hdfs.DataStreamer - Slow
> waitForAckedSeqno took 391210ms (threshold=3ms). File being written:
> /hbase/WALs/,60020,1626191371499/%2C60020%2C1626191371499.1627480615620,
> block: BP-958889176--1567030695029:blk_1689647875_616028364, Write
> pipeline datanodes:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]].
> 2021-07-28 14:19:30,589 WARN [sync.1] hdfs.DataStreamer - Slow
> waitForAckedSeqno took 391148ms (threshold=3ms). File being written:
> /hbase/WALs/,60020,1626191371499/%2C60020%2C1626191371499.1627480615620,
> block: BP-958889176--1567030695029:blk_1689647875_616028364, Write
> pipeline datanodes:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]].
> 2021-07-28 14:19:30,589 WARN [sync.2] hdfs.DataStreamer - Slow
> waitForAckedSeqno took 391147ms (threshold=3ms). File being written:
> /hbase/WALs/,60020,1626191371499/%2C60020%2C1626191371499.1627480615620,
> block: BP-958889176--1567030695029:blk_1689647875_616028364, Write
> pipeline datanodes:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]].
> 2021-07-28 14:19:30,591 INFO [sync.0] wal.FSHLog - Slow sync cost: 391289
> ms, current pipeline:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]]
> 2021-07-28 14:19:30,591 INFO [sync.1] wal.FSHLog - Slow sync cost: 391227
> ms, current pipeline:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]]
> 2021-07-28 14:19:30,591 WARN [sync.1] wal.FSHLog - Requesting log roll
> because we exceeded slow sync threshold; time=391227 ms, threshold=1 ms,
> current pipeline:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]]
> 2021-07
[GitHub] [hbase] huaxiangsun opened a new pull request #3668: HBASE-26255 Add an option to use region location from meta table in T…
huaxiangsun opened a new pull request #3668: URL: https://github.com/apache/hbase/pull/3668 …ableSnapshotInputFormat (#3661) Signed-off-by: Anoop Sam John -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-21521) Expose master startup status via JMX and web UI
[ https://issues.apache.org/jira/browse/HBASE-21521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412168#comment-17412168 ] Rushabh Shah edited comment on HBASE-21521 at 9/8/21, 8:17 PM: --- Raised a PR: [https://github.com/apache/hbase/pull/3667] Screenshots attached. Added a sleep for 30 seconds for master to become active, so that I can see that progress page is showing correct information. was (Author: shahrs87): Raised a PR: [https://github.com/apache/hbase/pull/3667] Screenshots attached. > Expose master startup status via JMX and web UI > --- > > Key: HBASE-21521 > URL: https://issues.apache.org/jira/browse/HBASE-21521 > Project: HBase > Issue Type: Improvement > Components: master, UI >Reporter: Andrew Kyle Purtell >Assignee: Rushabh Shah >Priority: Major > Attachments: hbase-21521-1.png, hbase-21521-2.png, hbase-21521-3.png, > hbase-21521-4.png > > > Add an internal API to the master for tracking startup progress. Expose this > information via JMX. > Modify the master to bring the web UI up sooner. Will require tweaks to > various views to prevent attempts to retrieve state before the master fully > up (or else expect NPEs). Currently, before the master has fully initialized > an attempt to use the web UI will return a 500 error code and display an > error page. > Finally, update the web UI to display startup progress, like HDFS-4249. > Filing this for branch-1. Need to check what if anything is available or > improved in branch-2 and master. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-21521) Expose master startup status via JMX and web UI
[ https://issues.apache.org/jira/browse/HBASE-21521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rushabh Shah updated HBASE-21521: - Attachment: hbase-21521-4.png hbase-21521-3.png hbase-21521-2.png hbase-21521-1.png > Expose master startup status via JMX and web UI > --- > > Key: HBASE-21521 > URL: https://issues.apache.org/jira/browse/HBASE-21521 > Project: HBase > Issue Type: Improvement > Components: master, UI >Reporter: Andrew Kyle Purtell >Assignee: Rushabh Shah >Priority: Major > Attachments: hbase-21521-1.png, hbase-21521-2.png, hbase-21521-3.png, > hbase-21521-4.png > > > Add an internal API to the master for tracking startup progress. Expose this > information via JMX. > Modify the master to bring the web UI up sooner. Will require tweaks to > various views to prevent attempts to retrieve state before the master fully > up (or else expect NPEs). Currently, before the master has fully initialized > an attempt to use the web UI will return a 500 error code and display an > error page. > Finally, update the web UI to display startup progress, like HDFS-4249. > Filing this for branch-1. Need to check what if anything is available or > improved in branch-2 and master. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-21521) Expose master startup status via JMX and web UI
[ https://issues.apache.org/jira/browse/HBASE-21521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412168#comment-17412168 ] Rushabh Shah commented on HBASE-21521: -- Raised a PR: [https://github.com/apache/hbase/pull/3667] Screenshots attached. > Expose master startup status via JMX and web UI > --- > > Key: HBASE-21521 > URL: https://issues.apache.org/jira/browse/HBASE-21521 > Project: HBase > Issue Type: Improvement > Components: master, UI >Reporter: Andrew Kyle Purtell >Assignee: Rushabh Shah >Priority: Major > > Add an internal API to the master for tracking startup progress. Expose this > information via JMX. > Modify the master to bring the web UI up sooner. Will require tweaks to > various views to prevent attempts to retrieve state before the master fully > up (or else expect NPEs). Currently, before the master has fully initialized > an attempt to use the web UI will return a 500 error code and display an > error page. > Finally, update the web UI to display startup progress, like HDFS-4249. > Filing this for branch-1. Need to check what if anything is available or > improved in branch-2 and master. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-26195) Data is present in replicated cluster but not present in primary cluster.
[
https://issues.apache.org/jira/browse/HBASE-26195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rushabh Shah updated HBASE-26195:
-
Fix Version/s: 1.8.0
> Data is present in replicated cluster but not present in primary cluster.
> -
>
> Key: HBASE-26195
> URL: https://issues.apache.org/jira/browse/HBASE-26195
> Project: HBase
> Issue Type: Bug
> Components: Replication, wal
>Affects Versions: 1.7.0
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Major
> Fix For: 1.8.0
>
>
> We encountered a case where we are seeing some rows (via Phoenix) in
> replicated cluster but they are not present in source/active cluster.
> Triaging further we found memstore rollback logs in few of the region servers.
> {noformat}
> 2021-07-28 14:17:59,353 DEBUG [3,queue=3,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,353 DEBUG [,queue=25,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,354 DEBUG [3,queue=3,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,354 DEBUG [,queue=25,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,354 DEBUG [3,queue=3,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,354 DEBUG [,queue=25,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,355 DEBUG [3,queue=3,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,355 DEBUG [,queue=25,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,356 DEBUG [,queue=25,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> {noformat}
> Looking more into logs, found that there were some hdfs layer issues sync'ing
> wal to hdfs.
> It was taking around 6 mins to sync wal. Logs below
> {noformat}
> 2021-07-28 14:19:30,511 WARN [sync.0] hdfs.DataStreamer - Slow
> waitForAckedSeqno took 391210ms (threshold=3ms). File being written:
> /hbase/WALs/,60020,1626191371499/%2C60020%2C1626191371499.1627480615620,
> block: BP-958889176--1567030695029:blk_1689647875_616028364, Write
> pipeline datanodes:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]].
> 2021-07-28 14:19:30,589 WARN [sync.1] hdfs.DataStreamer - Slow
> waitForAckedSeqno took 391148ms (threshold=3ms). File being written:
> /hbase/WALs/,60020,1626191371499/%2C60020%2C1626191371499.1627480615620,
> block: BP-958889176--1567030695029:blk_1689647875_616028364, Write
> pipeline datanodes:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]].
> 2021-07-28 14:19:30,589 WARN [sync.2] hdfs.DataStreamer - Slow
> waitForAckedSeqno took 391147ms (threshold=3ms). File being written:
> /hbase/WALs/,60020,1626191371499/%2C60020%2C1626191371499.1627480615620,
> block: BP-958889176--1567030695029:blk_1689647875_616028364, Write
> pipeline datanodes:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]].
> 2021-07-28 14:19:30,591 INFO [sync.0] wal.FSHLog - Slow sync cost: 391289
> ms, current pipeline:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]]
> 2021-07-28 14:19:30,591 INFO [sync.1] wal.FSHLog - Slow sync cost: 391227
> ms, current pipeline:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]]
> 2021-07-28 14:19:30,591 WARN [sync.1] wal.FSHLog - Requesting log roll
> because we exceeded slow sync threshold; time=391227 ms, threshold=1 ms,
> current pipeline:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]]
> 2021-0
[GitHub] [hbase] wchevreuil commented on pull request #3666: HBASE-26246 Persist the store engine configuration to TableDescriptor when creating table
wchevreuil commented on pull request #3666: URL: https://github.com/apache/hbase/pull/3666#issuecomment-915527033 > Oh, wait a minute, the title is to persist the store engine configuration, not only store file tracker. Oh yeah, missed that. Let me re-work it accordingly. > > For me, I do not think changing the global StoreEngine implementation should effect the existing tables, they should only effect newly created tables. > > WDYT @wchevreuil ? Yeah, that's what I was thinking, so only bothered about this in CreateTableProcedure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] shahrs87 commented on pull request #3611: [branch-1] HBASE-26195 Abort RS if wal sync fails or times out
shahrs87 commented on pull request #3611: URL: https://github.com/apache/hbase/pull/3611#issuecomment-915503572 Thank you @gjacoby126 for the review and commit ! Thank you @apurtell for the review ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] gjacoby126 merged pull request #3611: [branch-1] HBASE-26195 Abort RS if wal sync fails or times out
gjacoby126 merged pull request #3611: URL: https://github.com/apache/hbase/pull/3611 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2586: HBASE-25217 [Metrics] Add metrics for Call in IPC response queue
Apache-HBase commented on pull request #2586: URL: https://github.com/apache/hbase/pull/2586#issuecomment-915486013 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 6m 1s | master passed | | +1 :green_heart: | compile | 2m 3s | master passed | | +1 :green_heart: | shadedjars | 10m 53s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 15s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 42s | the patch passed | | +1 :green_heart: | compile | 2m 1s | the patch passed | | +1 :green_heart: | javac | 2m 1s | the patch passed | | +1 :green_heart: | shadedjars | 11m 18s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 13s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 47s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 163m 15s | hbase-server in the patch passed. | | | | 208m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2586/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2586 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 354cf2a8fdcb 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / fae6261d96 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2586/1/testReport/ | | Max. process+thread count | 3670 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2586/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26197) Fix some obvious bugs in MultiByteBuff.put
[
https://issues.apache.org/jira/browse/HBASE-26197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412136#comment-17412136
]
Hudson commented on HBASE-26197:
Results for branch branch-2
[build #340 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/340/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/340/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/340/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/340/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/340/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix some obvious bugs in MultiByteBuff.put
> --
>
> Key: HBASE-26197
> URL: https://issues.apache.org/jira/browse/HBASE-26197
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.4.5
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.3.7, 2.4.7
>
>
> MultiByteBuff.put(int destOffset, ByteBuff src, int srcOffset, int length)
> has some obvious bugs:
> * It seems mix up {{items}} in {{src}} {{MutiByteBuff}} and {{items}} in the
> {{dest}} {{MultiByteBuff}} , just as line 749 and line 754 illustrated. The
> logic is only right when src {{ByteBuff}} is also a {{MultiByteBuff}} and
> byte size of every {{ByteBuffer}} in {{src.items}} has exactly the same size
> as every {{ByteBuffer}} in the {{dest.items}},but looking the usage of this
> method in the hbase project, obviously the assumption is not right.
> {code:java}
> 746 public MultiByteBuff put(int offset, ByteBuff src, int srcOffset, int
> length) {
> 747 checkRefCount();
> 748 int destItemIndex = getItemIndex(offset);
> 749 int srcItemIndex = getItemIndex(srcOffset);
> 750 ByteBuffer destItem = this.items[destItemIndex];
> 751 offset = offset - this.itemBeginPos[destItemIndex];
> 752
> 753ByteBuffer srcItem = getItemByteBuffer(src, srcItemIndex);
> 754srcOffset = srcOffset - this.itemBeginPos[srcItemIndex];
> ...
> {code}
>
> * If src is {{SingleByteBuff}} and its remaining space is fewer than
> length，when remaining space is exhausted, this {{MultiByteBuff.put}} method
> would not throw any exception and continue to put src {{ByteBuff}} once again
> from position 0 because following {{MultiByteBuff.getItemByteBuffer}} ignores
> index paramter for {{SingleByteBuff}} . Obviously, this behavior is much
> strange and unexpected.
> {code:java}
>private static ByteBuffer getItemByteBuffer(ByteBuff buf, int index) {
> return (buf instanceof SingleByteBuff) ? buf.nioByteBuffers()[0]
>: ((MultiByteBuff) buf).items[index];
> }
>{code}
> Why seems tests is OK with too much bugs? Because in normal cases, we just
> use {{SingleByteBuff}} not {{MultiByteBuff}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2586: HBASE-25217 [Metrics] Add metrics for Call in IPC response queue
Apache-HBase commented on pull request #2586: URL: https://github.com/apache/hbase/pull/2586#issuecomment-915467078 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 54s | master passed | | +1 :green_heart: | compile | 1m 21s | master passed | | +1 :green_heart: | shadedjars | 8m 21s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 46s | the patch passed | | +1 :green_heart: | compile | 1m 20s | the patch passed | | +1 :green_heart: | javac | 1m 20s | the patch passed | | +1 :green_heart: | shadedjars | 8m 16s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 55s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 36s | hbase-hadoop-compat in the patch passed. | | -1 :x: | unit | 146m 54s | hbase-server in the patch failed. | | | | 180m 3s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2586/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2586 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9f83a1e269a4 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / fae6261d96 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2586/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2586/1/testReport/ | | Max. process+thread count | 3691 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2586/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell edited a comment on pull request #2663: HBASE-24637 - Reseek regression related to filter SKIP hinting
apurtell edited a comment on pull request #2663: URL: https://github.com/apache/hbase/pull/2663#issuecomment-915456344 > I have updated the PR with a new config 'hbase.switchto.next.bytes.read' . No new config. Repeating my earlier comment. What user is actually going to understand and set this? Figure it out without user intervention. Please don't merge until this is addressed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #2663: HBASE-24637 - Reseek regression related to filter SKIP hinting
apurtell commented on pull request #2663: URL: https://github.com/apache/hbase/pull/2663#issuecomment-915456344 > I have updated the PR with a new config 'hbase.switchto.next.bytes.read' . No new config. Repeating my earlier comment. What user is actually going to understand and set this? Figure it out without user intervention. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #3611: [branch-1] HBASE-26195 Abort RS if wal sync fails or times out
apurtell commented on pull request #3611: URL: https://github.com/apache/hbase/pull/3611#issuecomment-915451202 @gjacoby126 Are you good with the latest revision of this PR? I will merge in a few hours unless I hear otherwise. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bbeaudreault commented on pull request #3663: HBASE-26240 Set InterfaceAudience to Private for BalanceRequest$Builder
bbeaudreault commented on pull request #3663: URL: https://github.com/apache/hbase/pull/3663#issuecomment-915444266 @ndimiduk I hear you, but honestly I feel like there are pros and cons on both sides of that. If I break it out, I'd need to make BalanceResponse's constructor package-private at least, and the `org.apache.hadoop.hbase.client` package is quite large. I'd probably also get rid of the convenience `BalanceResponse.newBuilder()` since putting that in BalanceResponseBuilder would be sort of redundant. So we'd change the access pattern to `new BalanceResponseBuilder().build()`. Not a huge deal, but it's less intuitive and now there's an asymmetry with BalanceRequest. I could do the same with BalanceRequest, but that's just expanding non-intuitiveness. I could leave the asymmetry, but I do think there's some value there and I feel like inner Builder classes are sort of conventional. None of that is a non-starter, but I'm not sure if it's explicitly better than having the inner class be IA.Private, or even IA.Public. I can do it if you feel strongly about it, but just wanted to put that out there for consideration. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3666: HBASE-26246 Persist the store engine configuration to TableDescriptor when creating table
Apache-HBase commented on pull request #3666: URL: https://github.com/apache/hbase/pull/3666#issuecomment-915436093 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-26067 Compile Tests _ | | +1 :green_heart: | mvninstall | 7m 42s | HBASE-26067 passed | | +1 :green_heart: | compile | 2m 0s | HBASE-26067 passed | | +1 :green_heart: | shadedjars | 11m 48s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 4s | HBASE-26067 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 33s | the patch passed | | +1 :green_heart: | compile | 1m 41s | the patch passed | | +1 :green_heart: | javac | 1m 41s | the patch passed | | +1 :green_heart: | shadedjars | 10m 49s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 56s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 235m 37s | hbase-server in the patch failed. | | | | 281m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3666/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3666 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 94ef8b2957d7 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-26067 / 0c8db602a9 | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3666/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3666/1/testReport/ | | Max. process+thread count | 3498 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3666/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26197) Fix some obvious bugs in MultiByteBuff.put
[
https://issues.apache.org/jira/browse/HBASE-26197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412098#comment-17412098
]
Hudson commented on HBASE-26197:
Results for branch branch-2.3
[build #288 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/288/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/288/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/288/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/288/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/288/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix some obvious bugs in MultiByteBuff.put
> --
>
> Key: HBASE-26197
> URL: https://issues.apache.org/jira/browse/HBASE-26197
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.4.5
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.3.7, 2.4.7
>
>
> MultiByteBuff.put(int destOffset, ByteBuff src, int srcOffset, int length)
> has some obvious bugs:
> * It seems mix up {{items}} in {{src}} {{MutiByteBuff}} and {{items}} in the
> {{dest}} {{MultiByteBuff}} , just as line 749 and line 754 illustrated. The
> logic is only right when src {{ByteBuff}} is also a {{MultiByteBuff}} and
> byte size of every {{ByteBuffer}} in {{src.items}} has exactly the same size
> as every {{ByteBuffer}} in the {{dest.items}},but looking the usage of this
> method in the hbase project, obviously the assumption is not right.
> {code:java}
> 746 public MultiByteBuff put(int offset, ByteBuff src, int srcOffset, int
> length) {
> 747 checkRefCount();
> 748 int destItemIndex = getItemIndex(offset);
> 749 int srcItemIndex = getItemIndex(srcOffset);
> 750 ByteBuffer destItem = this.items[destItemIndex];
> 751 offset = offset - this.itemBeginPos[destItemIndex];
> 752
> 753ByteBuffer srcItem = getItemByteBuffer(src, srcItemIndex);
> 754srcOffset = srcOffset - this.itemBeginPos[srcItemIndex];
> ...
> {code}
>
> * If src is {{SingleByteBuff}} and its remaining space is fewer than
> length，when remaining space is exhausted, this {{MultiByteBuff.put}} method
> would not throw any exception and continue to put src {{ByteBuff}} once again
> from position 0 because following {{MultiByteBuff.getItemByteBuffer}} ignores
> index paramter for {{SingleByteBuff}} . Obviously, this behavior is much
> strange and unexpected.
> {code:java}
>private static ByteBuffer getItemByteBuffer(ByteBuff buf, int index) {
> return (buf instanceof SingleByteBuff) ? buf.nioByteBuffers()[0]
>: ((MultiByteBuff) buf).items[index];
> }
>{code}
> Why seems tests is OK with too much bugs? Because in normal cases, we just
> use {{SingleByteBuff}} not {{MultiByteBuff}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-26259) Fallback support to pure Java compression via aircompressor
[ https://issues.apache.org/jira/browse/HBASE-26259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-26259: Description: Airlift’s aircompressor (https://search.maven.org/artifact/io.airlift/aircompressor) is an Apache 2 licensed library, for Java 8 and up, available in Maven central, which provides pure Java implementations of gzip, lz4, lzo, snappy, and zstd and Hadoop compression codecs for same, claiming “_they are typically 300% faster than the JNI wrappers_.” (https://github.com/airlift/aircompressor). This library is under active development and up to date releases because it is used by Trino. Proposed changes: * Modify Compression.java such that compression codec implementation classes can be specified by configuration. Currently they are hardcoded as strings. * Pull in aircompressor as a ‘compile’ time dependency so it will be bundled into our build and made available on the server classpath. * Modify Compression.java to fall back to an aircompressor pure Java implementation if schema specifies a compression algorithm, a Hadoop native codec was specified as desired implementation, but the requisite native support is somehow not available. was: Airlift’s aircompressor (https://search.maven.org/artifact/io.airlift/aircompressor) is an Apache 2 licensed library, for Java 8 and up, available in Maven central, which provides both pure Java implementations of gzip, lz4, lzo, snappy, and zstd and Hadoop compression codecs for same, claiming “they are typically 300% faster than the JNI wrappers.” (https://github.com/airlift/aircompressor). This library is under active development and up to date releases because it is used by Trino. Proposed changes: * Modify Compression.java such that compression codec implementation classes can be specified by configuration. Currently they are hardcoded as strings. * Pull in aircompressor as a ‘compile’ time dependency so it will be bundled into our build and made available on the server classpath. * Modify Compression.java to fall back to an aircompressor pure Java implementation if schema specifies a compression algorithm, a Hadoop native codec was specified as desired implementation, but the requisite native support is somehow not available. > Fallback support to pure Java compression via aircompressor > > > Key: HBASE-26259 > URL: https://issues.apache.org/jira/browse/HBASE-26259 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-2 > > > Airlift’s aircompressor > (https://search.maven.org/artifact/io.airlift/aircompressor) is an Apache 2 > licensed library, for Java 8 and up, available in Maven central, which > provides pure Java implementations of gzip, lz4, lzo, snappy, and zstd and > Hadoop compression codecs for same, claiming “_they are typically 300% faster > than the JNI wrappers_.” (https://github.com/airlift/aircompressor). This > library is under active development and up to date releases because it is > used by Trino. > Proposed changes: > * Modify Compression.java such that compression codec implementation classes > can be specified by configuration. Currently they are hardcoded as strings. > * Pull in aircompressor as a ‘compile’ time dependency so it will be bundled > into our build and made available on the server classpath. > * Modify Compression.java to fall back to an aircompressor pure Java > implementation if schema specifies a compression algorithm, a Hadoop native > codec was specified as desired implementation, but the requisite native > support is somehow not available. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23834) HBase fails to run on Hadoop 3.3.0/3.2.2/3.1.4 due to jetty version mismatch
[ https://issues.apache.org/jira/browse/HBASE-23834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412077#comment-17412077 ] Nick Dimiduk commented on HBASE-23834: -- Thanks for the notification. I left an opinion on HBASE-26253. > HBase fails to run on Hadoop 3.3.0/3.2.2/3.1.4 due to jetty version mismatch > > > Key: HBASE-23834 > URL: https://issues.apache.org/jira/browse/HBASE-23834 > Project: HBase > Issue Type: Bug > Components: dependencies >Reporter: Wei-Chiu Chuang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > HBase master branch is currently on Jetty 9.3, and latest Hadoop 3 > (unreleased branches trunk, branch-3.2 and branch-3.1) bumped Jetty to 9.4 to > address a vulnerability CVE-2017-9735. > (1) Jetty 9.3 and 9.4 are quite different (there are incompatible API > changes) and HBase won't start on the latest Hadoop 3. > (2) In any case, HBase should update its Jetty dependency to address the > vulnerability. > Fortunately for HBase, updating to Jetty 9.4 requires no code change other > than the maven version string. > More tests are needed to verify if HBase can run on older Hadoop versions if > its Jetty is updated. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26253) Backport HBASE-23834 to branch 2.3
[ https://issues.apache.org/jira/browse/HBASE-26253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412076#comment-17412076 ] Nick Dimiduk commented on HBASE-26253: -- In terms of our compatibility guidelines, this is a big incompatible change, not appropriate for a patch release unless we're in extreme duress. 2.3.x is nearing the end of its life, and 2.4.x is well into its release line. Why should we make such a potentially disruptive change instead of EOLing 2.3? > Backport HBASE-23834 to branch 2.3 > -- > > Key: HBASE-26253 > URL: https://issues.apache.org/jira/browse/HBASE-26253 > Project: HBase > Issue Type: Bug >Affects Versions: 2.3.0, 2.3.1, 2.3.3, 2.3.2, 2.3.4, 2.3.5, 2.3.6 >Reporter: Shivam Garg >Priority: Major > Labels: patch > Fix For: 2.3.7 > > Attachments: HBASE-26253.branch-2.3.001.patch > > > HBase 2.3 branch is currently on Jetty 9.3, and latest Hadoop 3 (unreleased > branches trunk, branch-3.2 and branch-3.1) bumped Jetty to 9.4 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] huaxiangsun merged pull request #3661: HBASE-26255 Add an option to use region location from meta table in T…
huaxiangsun merged pull request #3661: URL: https://github.com/apache/hbase/pull/3661 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #3663: HBASE-26240 Set InterfaceAudience to Private for BalanceRequest$Builder
ndimiduk commented on pull request #3663: URL: https://github.com/apache/hbase/pull/3663#issuecomment-915394733 Sorry to nit-pick, but if the build is explicitly private API, can it be its own top-level class, rather than being an inner class of the public API? I think it's confusing to downstream developers when we inter-mingle public and private API resources in the same class. otherwise, +1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3665: HBASE-26248 Should find a suitable way to let users specify the store…
Apache-HBase commented on pull request #3665: URL: https://github.com/apache/hbase/pull/3665#issuecomment-915390839 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 6s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-26067 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 7s | HBASE-26067 passed | | +1 :green_heart: | compile | 1m 8s | HBASE-26067 passed | | +1 :green_heart: | shadedjars | 8m 42s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | HBASE-26067 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 42s | the patch passed | | +1 :green_heart: | compile | 1m 0s | the patch passed | | +1 :green_heart: | javac | 1m 0s | the patch passed | | +1 :green_heart: | shadedjars | 8m 9s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 157m 41s | hbase-server in the patch passed. | | | | 192m 19s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3665/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3665 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 462874439fcd 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-26067 / 0c8db602a9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3665/2/testReport/ | | Max. process+thread count | 3589 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3665/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2586: HBASE-25217 [Metrics] Add metrics for Call in IPC response queue
Apache-HBase commented on pull request #2586: URL: https://github.com/apache/hbase/pull/2586#issuecomment-915387230 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 1s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 9s | master passed | | +1 :green_heart: | compile | 4m 44s | master passed | | +1 :green_heart: | checkstyle | 1m 48s | master passed | | +1 :green_heart: | spotbugs | 3m 30s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 46s | the patch passed | | +1 :green_heart: | compile | 4m 39s | the patch passed | | +1 :green_heart: | javac | 4m 39s | the patch passed | | -0 :warning: | checkstyle | 0m 17s | hbase-hadoop-compat: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 25m 50s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 3m 1s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. | | | | 66m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2586/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2586 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux ffdb062fe01b 4.15.0-151-generic #157-Ubuntu SMP Fri Jul 9 23:07:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / fae6261d96 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2586/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-hadoop-compat.txt | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2586/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3665: HBASE-26248 Should find a suitable way to let users specify the store…
Apache-HBase commented on pull request #3665: URL: https://github.com/apache/hbase/pull/3665#issuecomment-915384942 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-26067 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 44s | HBASE-26067 passed | | +1 :green_heart: | compile | 1m 13s | HBASE-26067 passed | | +1 :green_heart: | shadedjars | 8m 31s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 45s | HBASE-26067 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 27s | the patch passed | | +1 :green_heart: | compile | 1m 16s | the patch passed | | +1 :green_heart: | javac | 1m 16s | the patch passed | | +1 :green_heart: | shadedjars | 8m 17s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 144m 34s | hbase-server in the patch passed. | | | | 177m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3665/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3665 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 968dc31496a7 4.15.0-151-generic #157-Ubuntu SMP Fri Jul 9 23:07:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-26067 / 0c8db602a9 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3665/2/testReport/ | | Max. process+thread count | 4701 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3665/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26245) Store region server list in master local region
[ https://issues.apache.org/jira/browse/HBASE-26245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412062#comment-17412062 ] Josh Elser commented on HBASE-26245: The ephemeral node (and notification from Watchers) are certainly nice from ZK, but it's interesting to start to think about using the Master Region for more "ownership" of our cluster membership. It will be a balancing act to figure out just how much ZK we want to remove :) The biggest problem when the WALs disappear is just that we don't know that we have SCP's that need to be scheduled (both for user and system tables -- for meta too). At a minimum, if we track RegionServers which are "dead but not yet processed" in the Master Region, I think that handles our biggest "we don't know if it's safe to schedule SCP" concern. At the same time, I'm wondering what other neat stuff we could store/learn from the RS's in here (like, would we want to hold on to stuff like the RegionServerStatusReport in this table?). Happy to help think through this some more, too! > Store region server list in master local region > --- > > Key: HBASE-26245 > URL: https://issues.apache.org/jira/browse/HBASE-26245 > Project: HBase > Issue Type: Brainstorming > Components: master, Zookeeper >Reporter: Duo Zhang >Priority: Major > > Just a simple idea still need to be polished. > For large clusters, ZooKeeper could be a bottle neck, there are some related > issues to not track region server list as much as possible, but what if we > want to do more, just do not register region server on zk? > I think, zookeeper here is act as something like a service registry, we need > the list to get all the region servers, and also need to know the changes of > the list. > But in fact, we could also kill region server from master side, so the latter > one could be done by a periodical heartbeat check daemon on master. And for > the former one, we could store the list in master local region, so when > master restart, it could setup the region server list by load from master > local region. > And there are mainly two other side effect which are good: > 1. We do not need to list the WAL directory on HDFS to find the previous > region servers, for scheduling SCP. This could make it possible to restart a > new HBase cluster based on only the root directory. > 2. For now, region server needs to register to HMaster first, and then put > its node on ZooKeeper, if it fails between these two actions, there is no way > for HMaster to clean this dead server, as it never expires on ZooKeeper. > There should be a related issue. If we just do not store a node on zk, then > this problem is also gone. > Of course, there will still be lots of other problems, like whether we need > another heartbeat call as reportForDuty is a bit heavy as we also report the > region list, etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26266) Improve developer docs for InterfaceAudience of public experimental APIs
Bryan Beaudreault created HBASE-26266: - Summary: Improve developer docs for InterfaceAudience of public experimental APIs Key: HBASE-26266 URL: https://issues.apache.org/jira/browse/HBASE-26266 Project: HBase Issue Type: Task Reporter: Bryan Beaudreault Per [https://lists.apache.org/thread.html/re7736008a7619029bf34ed81f5931c537281d22788719f2dcf37f0c0%40%3Cdev.hbase.apache.org%3E,] our InterfaceAudience.Public restrictions intentionally deviate from the default yetus docs, and we would like to keep it that way. It was acknowledged that there may be a reason to expose early access to certain APIs that may need to evolve in breaking ways. The suggested approach is to annotate these as InterfaceAudience.LimitedPrivate. We should update developer docs to clarify both points. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26265) Update ref guide to mention the new store file tracker implementations
Duo Zhang created HBASE-26265: - Summary: Update ref guide to mention the new store file tracker implementations Key: HBASE-26265 URL: https://issues.apache.org/jira/browse/HBASE-26265 Project: HBase Issue Type: Sub-task Components: documentation Reporter: Duo Zhang For example, when to use these store file trackers. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-26264) Add more checks to prevent misconfiguration on store file tracker
[ https://issues.apache.org/jira/browse/HBASE-26264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26264: -- Description: For example, when creating new tables, if the store file tracker is configured to MigrationStoreFileTracker, we should fail the creation and tell users you should not use this store file tracker for a new table. And when switching the implementation for store file tracker, we should let end users first switch to MigrationStoreFileTracker, and then to the final target store file tracker. If end users configure wrong store file tracker, we should fail the alter table operation. > Add more checks to prevent misconfiguration on store file tracker > - > > Key: HBASE-26264 > URL: https://issues.apache.org/jira/browse/HBASE-26264 > Project: HBase > Issue Type: Sub-task > Components: conf, HFile >Reporter: Duo Zhang >Priority: Major > > For example, when creating new tables, if the store file tracker is > configured to MigrationStoreFileTracker, we should fail the creation and tell > users you should not use this store file tracker for a new table. > And when switching the implementation for store file tracker, we should let > end users first switch to MigrationStoreFileTracker, and then to the final > target store file tracker. If end users configure wrong store file tracker, > we should fail the alter table operation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26264) Add more checks to prevent misconfiguration on store file tracker
Duo Zhang created HBASE-26264: - Summary: Add more checks to prevent misconfiguration on store file tracker Key: HBASE-26264 URL: https://issues.apache.org/jira/browse/HBASE-26264 Project: HBase Issue Type: Sub-task Components: conf, HFile Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26228) updateRSGroupConfig operation should be authorized by AccessController
[ https://issues.apache.org/jira/browse/HBASE-26228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412005#comment-17412005 ] Pankaj Kumar commented on HBASE-26228: -- Thank you so much [~zhangduo] & [~zhaobaiqiang] for reviewing and merging the PR. > updateRSGroupConfig operation should be authorized by AccessController > --- > > Key: HBASE-26228 > URL: https://issues.apache.org/jira/browse/HBASE-26228 > Project: HBase > Issue Type: Bug > Components: acl, rsgroup >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Major > Fix For: 3.0.0-alpha-2 > > > Currently there is no ACL check for Admin#updateRSGroupConfig() operation. > AccessController should implement MasterObserver#preUpdateRSGroupConfig(). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3664: HBASE-26261 Store configuration loss when use update_config
Apache-HBase commented on pull request #3664: URL: https://github.com/apache/hbase/pull/3664#issuecomment-915310560 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 7s | master passed | | +1 :green_heart: | compile | 1m 16s | master passed | | +1 :green_heart: | shadedjars | 10m 14s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 47s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 46s | the patch passed | | +1 :green_heart: | compile | 1m 15s | the patch passed | | +1 :green_heart: | javac | 1m 15s | the patch passed | | +1 :green_heart: | shadedjars | 10m 13s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 242m 49s | hbase-server in the patch passed. | | | | 280m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3664/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3664 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 404fec0a62d6 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / fae6261d96 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3664/1/testReport/ | | Max. process+thread count | 2897 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3664/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3664: HBASE-26261 Store configuration loss when use update_config
Apache-HBase commented on pull request #3664: URL: https://github.com/apache/hbase/pull/3664#issuecomment-915308236 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 58s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 14s | master passed | | +1 :green_heart: | compile | 1m 20s | master passed | | +1 :green_heart: | shadedjars | 9m 10s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 45s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 52s | the patch passed | | +1 :green_heart: | compile | 1m 19s | the patch passed | | +1 :green_heart: | javac | 1m 19s | the patch passed | | +1 :green_heart: | shadedjars | 9m 3s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 243m 31s | hbase-server in the patch failed. | | | | 279m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3664/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3664 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9dcab2dab3e6 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / fae6261d96 | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3664/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3664/1/testReport/ | | Max. process+thread count | 3479 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3664/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3665: HBASE-26248 Should find a suitable way to let users specify the store…
Apache-HBase commented on pull request #3665: URL: https://github.com/apache/hbase/pull/3665#issuecomment-915274421 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-26067 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 14s | HBASE-26067 passed | | +1 :green_heart: | compile | 3m 30s | HBASE-26067 passed | | +1 :green_heart: | checkstyle | 1m 5s | HBASE-26067 passed | | +1 :green_heart: | spotbugs | 2m 7s | HBASE-26067 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 0s | the patch passed | | +1 :green_heart: | compile | 3m 20s | the patch passed | | +1 :green_heart: | javac | 3m 20s | the patch passed | | +1 :green_heart: | checkstyle | 1m 5s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 46s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 22s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 50m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3665/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3665 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 55f0e35551ed 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-26067 / 0c8db602a9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3665/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #3664: HBASE-26261 Store configuration loss when use update_config
Apache9 commented on a change in pull request #3664:
URL: https://github.com/apache/hbase/pull/3664#discussion_r704419809
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
##
@@ -2529,6 +2529,8 @@ protected OffPeakHours getOffPeakHours() {
public void onConfigurationChange(Configuration conf) {
this.conf = new CompoundConfiguration()
.add(conf)
+.addBytesMap(region.getTableDescriptor().getValues())
Review comment:
Better abstract a util method to do the merging of the configurations
and call it from both the constructor and here?
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerOnlineConfigChange.java
##
@@ -259,4 +263,12 @@ public void removeClosedRegionFromConfigurationManager()
throws Exception {
});
}
}
+
+ @Test
+ public void testStoreConfigurationOnlineChange() {
+rs1.getConfigurationManager().notifyAllObservers(conf);
+long actualMaxFileSize =
+
r1.getStore(COLUMN_FAMILY1).conf.getLong(TableDescriptorBuilder.MAX_FILESIZE,
-1);
Review comment:
Strange that we do not provide getConfiguration method for Region and
Store interface...
Let's open another issue to address it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-26163) Better logging in RSGroupInfoManagerImpl
[
https://issues.apache.org/jira/browse/HBASE-26163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17411938#comment-17411938
]
Hudson commented on HBASE-26163:
Results for branch master
[build #387 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/387/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/387/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/387/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/387/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Better logging in RSGroupInfoManagerImpl
>
>
> Key: HBASE-26163
> URL: https://issues.apache.org/jira/browse/HBASE-26163
> Project: HBase
> Issue Type: Improvement
> Components: logging, rsgroup
>Reporter: Caroline Zhou
>Assignee: Caroline Zhou
>Priority: Minor
> Fix For: 3.0.0-alpha-2
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26228) updateRSGroupConfig operation should be authorized by AccessController
[
https://issues.apache.org/jira/browse/HBASE-26228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17411937#comment-17411937
]
Hudson commented on HBASE-26228:
Results for branch master
[build #387 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/387/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/387/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/387/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/387/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> updateRSGroupConfig operation should be authorized by AccessController
> ---
>
> Key: HBASE-26228
> URL: https://issues.apache.org/jira/browse/HBASE-26228
> Project: HBase
> Issue Type: Bug
> Components: acl, rsgroup
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Major
> Fix For: 3.0.0-alpha-2
>
>
> Currently there is no ACL check for Admin#updateRSGroupConfig() operation.
> AccessController should implement MasterObserver#preUpdateRSGroupConfig().
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26197) Fix some obvious bugs in MultiByteBuff.put
[
https://issues.apache.org/jira/browse/HBASE-26197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17411939#comment-17411939
]
Hudson commented on HBASE-26197:
Results for branch master
[build #387 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/387/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/387/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/387/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/387/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix some obvious bugs in MultiByteBuff.put
> --
>
> Key: HBASE-26197
> URL: https://issues.apache.org/jira/browse/HBASE-26197
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.4.5
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.3.7, 2.4.7
>
>
> MultiByteBuff.put(int destOffset, ByteBuff src, int srcOffset, int length)
> has some obvious bugs:
> * It seems mix up {{items}} in {{src}} {{MutiByteBuff}} and {{items}} in the
> {{dest}} {{MultiByteBuff}} , just as line 749 and line 754 illustrated. The
> logic is only right when src {{ByteBuff}} is also a {{MultiByteBuff}} and
> byte size of every {{ByteBuffer}} in {{src.items}} has exactly the same size
> as every {{ByteBuffer}} in the {{dest.items}},but looking the usage of this
> method in the hbase project, obviously the assumption is not right.
> {code:java}
> 746 public MultiByteBuff put(int offset, ByteBuff src, int srcOffset, int
> length) {
> 747 checkRefCount();
> 748 int destItemIndex = getItemIndex(offset);
> 749 int srcItemIndex = getItemIndex(srcOffset);
> 750 ByteBuffer destItem = this.items[destItemIndex];
> 751 offset = offset - this.itemBeginPos[destItemIndex];
> 752
> 753ByteBuffer srcItem = getItemByteBuffer(src, srcItemIndex);
> 754srcOffset = srcOffset - this.itemBeginPos[srcItemIndex];
> ...
> {code}
>
> * If src is {{SingleByteBuff}} and its remaining space is fewer than
> length，when remaining space is exhausted, this {{MultiByteBuff.put}} method
> would not throw any exception and continue to put src {{ByteBuff}} once again
> from position 0 because following {{MultiByteBuff.getItemByteBuffer}} ignores
> index paramter for {{SingleByteBuff}} . Obviously, this behavior is much
> strange and unexpected.
> {code:java}
>private static ByteBuffer getItemByteBuffer(ByteBuff buf, int index) {
> return (buf instanceof SingleByteBuff) ? buf.nioByteBuffers()[0]
>: ((MultiByteBuff) buf).items[index];
> }
>{code}
> Why seems tests is OK with too much bugs? Because in normal cases, we just
> use {{SingleByteBuff}} not {{MultiByteBuff}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3666: HBASE-26246 Persist the store engine configuration to TableDescriptor when creating table
Apache-HBase commented on pull request #3666: URL: https://github.com/apache/hbase/pull/3666#issuecomment-915247784 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-26067 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 8s | HBASE-26067 passed | | +1 :green_heart: | compile | 3m 17s | HBASE-26067 passed | | +1 :green_heart: | checkstyle | 1m 1s | HBASE-26067 passed | | +1 :green_heart: | spotbugs | 2m 8s | HBASE-26067 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 44s | the patch passed | | +1 :green_heart: | compile | 3m 13s | the patch passed | | +1 :green_heart: | javac | 3m 13s | the patch passed | | +1 :green_heart: | checkstyle | 1m 4s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 21m 27s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 51m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3666/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3666 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 0758e5b2d837 4.15.0-151-generic #157-Ubuntu SMP Fri Jul 9 23:07:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-26067 / 0c8db602a9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3666/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26263) [Rolling Upgrading] Persist the StoreEngine configurations to TableDescriptor for existing tables
[ https://issues.apache.org/jira/browse/HBASE-26263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26263: -- Component/s: master HFile > [Rolling Upgrading] Persist the StoreEngine configurations to TableDescriptor > for existing tables > - > > Key: HBASE-26263 > URL: https://issues.apache.org/jira/browse/HBASE-26263 > Project: HBase > Issue Type: Sub-task > Components: HFile, master >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-26246) Persist the store engine configuration to TableDescriptor when creating table
[ https://issues.apache.org/jira/browse/HBASE-26246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26246: -- Component/s: master > Persist the store engine configuration to TableDescriptor when creating table > - > > Key: HBASE-26246 > URL: https://issues.apache.org/jira/browse/HBASE-26246 > Project: HBase > Issue Type: Sub-task > Components: HFile, master >Reporter: Duo Zhang >Assignee: Wellington Chevreuil >Priority: Major > > As discussed in this section in the design doc: > https://docs.google.com/document/d/16Nr1Fn3VaXuz1g1FTiME-bnGR3qVK5B-raXshOkDLcY/edit#heading=h.78r2mdeyquug > If we use different SFT implementation at master side and region server side, > it is likely to cause data loss, which is a very serious misconfiguration > problem. > A possible solution is to make sure that master and region server always load > the configurations about StoreEngine from the same place. To archive this, a > possible way is to always set the StoreEngine configurations to the > TableDescriptor, even if user does not explicitly set it when creating a > table. > And also, when upgrading, we should check whether the existing tables have > StoreEngine configurations, if not, we need to set them. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26263) [Rolling Upgrading] Persist the StoreEngine configurations to TableDescriptor for existing tables
Duo Zhang created HBASE-26263: - Summary: [Rolling Upgrading] Persist the StoreEngine configurations to TableDescriptor for existing tables Key: HBASE-26263 URL: https://issues.apache.org/jira/browse/HBASE-26263 Project: HBase Issue Type: Sub-task Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3666: HBASE-26246 Persist the store engine configuration to TableDescriptor when creating table
Apache-HBase commented on pull request #3666: URL: https://github.com/apache/hbase/pull/3666#issuecomment-915238252 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-26067 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 6s | HBASE-26067 passed | | +1 :green_heart: | compile | 1m 4s | HBASE-26067 passed | | +1 :green_heart: | shadedjars | 8m 19s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | HBASE-26067 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 1s | the patch passed | | +1 :green_heart: | compile | 1m 3s | the patch passed | | +1 :green_heart: | javac | 1m 3s | the patch passed | | +1 :green_heart: | shadedjars | 8m 18s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 8m 53s | hbase-server in the patch failed. | | | | 38m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3666/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3666 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1bf8fe84e47c 4.15.0-151-generic #157-Ubuntu SMP Fri Jul 9 23:07:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-26067 / 0c8db602a9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3666/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3666/1/testReport/ | | Max. process+thread count | 772 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3666/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #3666: HBASE-26246 Persist the store engine configuration to TableDescriptor when creating table
Apache9 commented on pull request #3666: URL: https://github.com/apache/hbase/pull/3666#issuecomment-915237785 Oh, wait a minute, the title is to persist the store engine configuration, not only store file tracker. For me, I do not think changing the global StoreEngine implementation should effect the existing tables, they should only effect newly created tables. WDYT @wchevreuil ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 edited a comment on pull request #3656: HBASE-26224 Introduce a MigrationStoreFileTracker to support migratin…
Apache9 edited a comment on pull request #3656: URL: https://github.com/apache/hbase/pull/3656#issuecomment-915229619 Any other concerns? @wchevreuil -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #3656: HBASE-26224 Introduce a MigrationStoreFileTracker to support migratin…
Apache9 commented on pull request #3656: URL: https://github.com/apache/hbase/pull/3656#issuecomment-915229619 Any other concerns? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #3666: HBASE-26246 Persist the store engine configuration to TableDescriptor when creating table
Apache9 commented on a change in pull request #3666:
URL: https://github.com/apache/hbase/pull/3666#discussion_r704406135
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateTableProcedure.java
##
@@ -285,6 +290,13 @@ private void preCreate(final MasterProcedureEnv env)
getTableName(), (newRegions != null ? newRegions.size() : 0));
}
+if(StringUtils.isEmpty(tableDescriptor.getValue(TRACK_IMPL))){
Review comment:
Seems the formatter file in your IDE is bit strange? Usually we will
have a space between 'if' and '('
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3665: HBASE-26248 Should find a suitable way to let users specify the store…
Apache-HBase commented on pull request #3665: URL: https://github.com/apache/hbase/pull/3665#issuecomment-915211884 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-26067 Compile Tests _ | | +1 :green_heart: | mvninstall | 7m 20s | HBASE-26067 passed | | +1 :green_heart: | compile | 1m 52s | HBASE-26067 passed | | +1 :green_heart: | shadedjars | 13m 10s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 12s | HBASE-26067 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 49s | the patch passed | | +1 :green_heart: | compile | 1m 49s | the patch passed | | +1 :green_heart: | javac | 1m 49s | the patch passed | | +1 :green_heart: | shadedjars | 13m 30s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 18s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 19m 57s | hbase-server in the patch failed. | | | | 71m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3665/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3665 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 273b903b475f 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-26067 / 0c8db602a9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3665/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3665/1/testReport/ | | Max. process+thread count | 771 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3665/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-26246) Persist the store engine configuration to TableDescriptor when creating table
[ https://issues.apache.org/jira/browse/HBASE-26246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil reassigned HBASE-26246: Assignee: Wellington Chevreuil > Persist the store engine configuration to TableDescriptor when creating table > - > > Key: HBASE-26246 > URL: https://issues.apache.org/jira/browse/HBASE-26246 > Project: HBase > Issue Type: Sub-task > Components: HFile >Reporter: Duo Zhang >Assignee: Wellington Chevreuil >Priority: Major > > As discussed in this section in the design doc: > https://docs.google.com/document/d/16Nr1Fn3VaXuz1g1FTiME-bnGR3qVK5B-raXshOkDLcY/edit#heading=h.78r2mdeyquug > If we use different SFT implementation at master side and region server side, > it is likely to cause data loss, which is a very serious misconfiguration > problem. > A possible solution is to make sure that master and region server always load > the configurations about StoreEngine from the same place. To archive this, a > possible way is to always set the StoreEngine configurations to the > TableDescriptor, even if user does not explicitly set it when creating a > table. > And also, when upgrading, we should check whether the existing tables have > StoreEngine configurations, if not, we need to set them. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3665: HBASE-26248 Should find a suitable way to let users specify the store…
Apache-HBase commented on pull request #3665: URL: https://github.com/apache/hbase/pull/3665#issuecomment-915202484 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 24s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-26067 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 46s | HBASE-26067 passed | | +1 :green_heart: | compile | 3m 43s | HBASE-26067 passed | | +1 :green_heart: | checkstyle | 1m 10s | HBASE-26067 passed | | +1 :green_heart: | spotbugs | 2m 24s | HBASE-26067 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 19s | the patch passed | | +1 :green_heart: | compile | 3m 41s | the patch passed | | +1 :green_heart: | javac | 3m 41s | the patch passed | | +1 :green_heart: | checkstyle | 1m 13s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 21m 59s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 17s | The patch does not generate ASF License warnings. | | | | 58m 18s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3665/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3665 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux f02bff8da50b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-26067 / 0c8db602a9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3665/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bbeaudreault commented on pull request #3660: HBASE-26252: Add option to reload balancer configs when invoking the balancer
bbeaudreault commented on pull request #3660: URL: https://github.com/apache/hbase/pull/3660#issuecomment-915195934 Test failure looks unrelated to this change: TestRSGroupsWithACL (AccessDenied) and TestJMXListener (Shutting down) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3656: HBASE-26224 Introduce a MigrationStoreFileTracker to support migratin…
Apache-HBase commented on pull request #3656: URL: https://github.com/apache/hbase/pull/3656#issuecomment-915166783 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-26067 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 0s | HBASE-26067 passed | | +1 :green_heart: | compile | 3m 10s | HBASE-26067 passed | | +1 :green_heart: | checkstyle | 1m 3s | HBASE-26067 passed | | +1 :green_heart: | spotbugs | 2m 7s | HBASE-26067 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 39s | the patch passed | | +1 :green_heart: | compile | 3m 10s | the patch passed | | +1 :green_heart: | javac | 3m 10s | the patch passed | | +1 :green_heart: | checkstyle | 1m 0s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 25s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 48m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3656/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3656 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 6f2ec7b56e85 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-26067 / 0c8db602a9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3656/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 opened a new pull request #3665: HBASE-26248 Should find a suitable way to let users specify the store…
Apache9 opened a new pull request #3665: URL: https://github.com/apache/hbase/pull/3665 … file tracker implementation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Work started] (HBASE-26248) Should find a suitable way to let users specify the store file tracker implementation
[ https://issues.apache.org/jira/browse/HBASE-26248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-26248 started by Duo Zhang. - > Should find a suitable way to let users specify the store file tracker > implementation > - > > Key: HBASE-26248 > URL: https://issues.apache.org/jira/browse/HBASE-26248 > Project: HBase > Issue Type: Sub-task > Components: HFile >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > Now all the implementations are marked as IA.Private, so there is no safe way > for users to specify them. > But for simplify, maybe we should not expose the full class name of the > implementation classes to users. Just follow the way in WALFactory, to > introduce an alias name for the implemention, for example > DEFAULT, FILE, REGION, etc. > Thoughts? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-26248) Should find a suitable way to let users specify the store file tracker implementation
[ https://issues.apache.org/jira/browse/HBASE-26248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-26248: - Assignee: Duo Zhang > Should find a suitable way to let users specify the store file tracker > implementation > - > > Key: HBASE-26248 > URL: https://issues.apache.org/jira/browse/HBASE-26248 > Project: HBase > Issue Type: Sub-task > Components: HFile >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > Now all the implementations are marked as IA.Private, so there is no safe way > for users to specify them. > But for simplify, maybe we should not expose the full class name of the > implementation classes to users. Just follow the way in WALFactory, to > introduce an alias name for the implemention, for example > DEFAULT, FILE, REGION, etc. > Thoughts? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3664: HBASE-26261 Store configuration loss when use update_config
Apache-HBase commented on pull request #3664: URL: https://github.com/apache/hbase/pull/3664#issuecomment-915134988 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 4s | master passed | | +1 :green_heart: | compile | 3m 19s | master passed | | +1 :green_heart: | checkstyle | 1m 3s | master passed | | +1 :green_heart: | spotbugs | 2m 4s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 38s | the patch passed | | +1 :green_heart: | compile | 3m 12s | the patch passed | | +1 :green_heart: | javac | 3m 12s | the patch passed | | +1 :green_heart: | checkstyle | 1m 1s | the patch passed | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 12s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 15s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 47m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3664/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3664 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux aaa6318b04dd 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / fae6261d96 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3664/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26262) Implement a region based store file tracker
[ https://issues.apache.org/jira/browse/HBASE-26262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17411838#comment-17411838 ] Duo Zhang commented on HBASE-26262: --- [~wchevreuil] FYI. I think you could try to move the region based tracking code from HBASE-24749 here. Thanks. > Implement a region based store file tracker > --- > > Key: HBASE-26262 > URL: https://issues.apache.org/jira/browse/HBASE-26262 > Project: HBase > Issue Type: Sub-task > Components: HFile >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
