[jira] [Created] (HBASE-26857) 1.开启rsgroup

范建波 created HBASE-26857: --- Summary: 1.开启rsgroup Key: HBASE-26857 URL: https://issues.apache.org/jira/browse/HBASE-26857 Project: HBase Issue Type: Bug Affects Versions: 2.1.0 Reporter: 范

[jira] [Created] (HBASE-26856) BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted

Mohammad Arshad created HBASE-26856: --- Summary: BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted Key: HBASE-26856 URL: https://issues.apache.org/jira/browse/HBASE-26856 Project: HBa

Re: [DISCUSS] HBASE-26849 NPE caused by WAL Compression and Replication

Hi Andrew, Thank you for sharing your detailed views. Originally my idea was to try to do a complete fix to this issue based on the existing implementation after working through this phase. But as we discussed before, LRUCache cannot be rolled back, so this may come at the cost of performance (ca

Re: [DISCUSS] HBASE-26849 NPE caused by WAL Compression and Replication

BTW, I think this issue might also be helpful for HBASE-15983. 唐天航 于2022年3月17日周四 10:33写道： > Hi xiaolin, > > I have read HBASE-26843 just now. > > If you use WAL Compression, I think this issue might be related to > HBASE-26849. > When you get a NullPointerException or IndexOutOfBoundsException f

Re: [DISCUSS] HBASE-26849 NPE caused by WAL Compression and Replication

Hi xiaolin, I have read HBASE-26843 just now. If you use WAL Compression, I think this issue might be related to HBASE-26849. When you get a NullPointerException or IndexOutOfBoundsException from LRUDictionary, ProtobufLogReader will wrap it as an EOFException and try to roll back the position an

[jira] [Reopened] (HBASE-26848) Set java.io.tmpdir on mvn command when running jenkins job

[ https://issues.apache.org/jira/browse/HBASE-26848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-26848: --- Need an addendum as the -Djava.io.tmpdir will override the one we set in surefire systemPropertiesVaria

[jira] [Resolved] (HBASE-26828) Increase the concurrency when running UTs in pre commit job

[ https://issues.apache.org/jira/browse/HBASE-26828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26828. --- Hadoop Flags: Reviewed Resolution: Fixed The problem has been solved, although there are still

Re: [DISCUSS] How to expose configs for users to tune

Thanks for your detailed response, Nick! > I think that none of my comments address your intended topic: how do we publish our configuration points as an API that can be consumed by user applications? (Do I have that correct?) This is a good summary, and I appreciate the other thoughts/clarificat

Re: [DISCUSS] HBASE-26849 NPE caused by WAL Compression and Replication

I wish email had an edit button. Let me rephrase I think what we need is an intermediate buffer that the reader fills with the WALedit contents. We read from the WAL stream into the buffer. If the reader is reset, the buffer is reset. Once the reader fully reads in a WALedit, it would pass the com

Re: [DISCUSS] HBASE-26849 NPE caused by WAL Compression and Replication

When I did the WAL value compression work (HBASE-25869) I tested the resulting code with an integration scenario with replication active. I initially had an issue with unrecoverable errors during rewind, but it was due to an error in my implementation and was corrected (HBASE-25994). After that the

Re: [NOTICE] The pre commit job is a bit unstable than before

+1 and thanks for fixing it. -Stephen On Wed, Mar 16, 2022 at 4:17 AM 张铎(Duo Zhang) wrote: > > The nightly jobs are executed on the same machine with pre commits, so they > will be effected too :) > > Nick Dimiduk 于2022年3月16日周三 18:42写道： > > > I think it's not just, nightlies are also more flake

[jira] [Created] (HBASE-26855) Delete unnecessary dependency on jaxb-runtime jar

Nick Dimiduk created HBASE-26855: Summary: Delete unnecessary dependency on jaxb-runtime jar Key: HBASE-26855 URL: https://issues.apache.org/jira/browse/HBASE-26855 Project: HBase Issue Type:

[jira] [Resolved] (HBASE-26813) Remove javax.ws.rs-api dependency

[ https://issues.apache.org/jira/browse/HBASE-26813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-26813. -- Resolution: Fixed > Remove javax.ws.rs-api dependency > - > >

[jira] [Created] (HBASE-26854) Shell startup logs a bunch of noise

Nick Dimiduk created HBASE-26854: Summary: Shell startup logs a bunch of noise Key: HBASE-26854 URL: https://issues.apache.org/jira/browse/HBASE-26854 Project: HBase Issue Type: Bug Affec

[jira] [Created] (HBASE-26853) Nightlies should test our gateway services

Nick Dimiduk created HBASE-26853: Summary: Nightlies should test our gateway services Key: HBASE-26853 URL: https://issues.apache.org/jira/browse/HBASE-26853 Project: HBase Issue Type: Task

Re: [DISCUSS] HBASE-26849 NPE caused by WAL Compression and Replication

No problem, I will read the details about HBASE-26843 tomorrow. And i think you could try HBASE-26849, but after all, my suggestion is close WAL Compression if you need to use replication... Xiaolin Ha 于2022年3月16日周三 22:40写道： > We have also used replication+wal compression in our production clust

Re: [DISCUSS] How to expose configs for users to tune

Hi Bryan, Thanks for bringing this up. I agree with Duo (and I think we have this settled as project-wide consensus) that HConstants is/was an anti-pattern, that we are actively against adding new fields there, and opportunistically removing fields when we can. Further, the documented meaning of

Re: [DISCUSS] HBASE-26849 NPE caused by WAL Compression and Replication

We have also used replication+wal compression in our production clusters. It really seems unstable when replicating compressed cells. I'm digging with the log queue overstock issues, and pointed out one in issue HBASE-26843, which might be related to HBASE-26849. We can discuss more details if you

[jira] [Reopened] (HBASE-26828) Increase the concurrency when running UTs in pre commit job

[ https://issues.apache.org/jira/browse/HBASE-26828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-26828: --- For adding addendum. See HBASE-26852. > Increase the concurrency when running UTs in pre commit job > -

[jira] [Created] (HBASE-26852) --hadoop-profile is not honored in branch-2 PR builds

Nick Dimiduk created HBASE-26852: Summary: --hadoop-profile is not honored in branch-2 PR builds Key: HBASE-26852 URL: https://issues.apache.org/jira/browse/HBASE-26852 Project: HBase Issue T

[jira] [Created] (HBASE-26851) [Documentation] Point out compatibility issues with WAL Compression and Replication in doc

tianhang tang created HBASE-26851: - Summary: [Documentation] Point out compatibility issues with WAL Compression and Replication in doc Key: HBASE-26851 URL: https://issues.apache.org/jira/browse/HBASE-26851

[jira] [Resolved] (HBASE-26275) update error message when executing deleteall with ROWPREFIXFILTER in meta table

[ https://issues.apache.org/jira/browse/HBASE-26275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaolin Ha resolved HBASE-26275. Resolution: Fixed Merged to branch-2.4+, thanks [~xichaomin]  for contributing. > update error me

[jira] [Resolved] (HBASE-26837) Set SFT config when creating TableDescriptor in TestCloneSnapshotProcedure

[ https://issues.apache.org/jira/browse/HBASE-26837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26837. --- Hadoop Flags: Reviewed Resolution: Fixed Merged to master and branch-2. Thanks [~Xiaolin Ha]

[jira] [Resolved] (HBASE-26848) Set java.io.tmpdir on mvn command when running jenkins job

[ https://issues.apache.org/jira/browse/HBASE-26848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26848. --- Fix Version/s: 2.5.0 1.7.2 3.0.0-alpha-3 2.4

Re: [NOTICE] The pre commit job is a bit unstable than before

The nightly jobs are executed on the same machine with pre commits, so they will be effected too :) Nick Dimiduk 于2022年3月16日周三 18:42写道： > I think it's not just, nightlies are also more flakey than usual. > > Thank you for being proactive on this, Duo (and others)! > > On Tue, Mar 15, 2022 at 2:2

Re: [NOTICE] The pre commit job is a bit unstable than before

I think it's not just, nightlies are also more flakey than usual. Thank you for being proactive on this, Duo (and others)! On Tue, Mar 15, 2022 at 2:29 PM 张铎(Duo Zhang) wrote: > After switching to ci-hbase, our pre commit job runs much slower than > before. > > This is because the new hbase jen

Re: [DISCUSS] HBASE-26849 NPE caused by WAL Compression and Replication

+1 on updating doc first. You can file an issue for the documentation change, and let's also send an NOTICE email to both dev and user list to warn our users about this. 唐天航 于2022年3月16日周三 18:08写道： > If we only reset the position to the head, yes we can fix it. > In fact, 26849 is to fix the prob

Re: [DISCUSS] HBASE-26849 NPE caused by WAL Compression and Replication

> > Maybe we need to do some refactoring... > I can not agree more... But before that, I think we'd better point out this compatibility issue explicitly in our doc. 唐天航 于2022年3月16日周三 18:08写道： > If we only reset the position to the head, yes we can fix it. > In fact, 26849 is to fix the problem i

Re: [DISCUSS] HBASE-26849 NPE caused by WAL Compression and Replication

If we only reset the position to the head, yes we can fix it. In fact, 26849 is to fix the problem in this scenario. But unfortunately, we have some other scenarios where we roll back the position to some intermediate position, such as ProtobufLogReader.java#L381

Re: [DISCUSS] HBASE-26849 NPE caused by WAL Compression and Replication

The old WAL compression implementation is buggy when used together with replication, that's true... But in general I think it is fixable, the dict is per file IIRC, so I think clearing the LRUCache when resetting to the head of the file can fix the problem? Maybe we need to do some refactoring...

Re: [VOTE] Second release candidate for HBase 2.4.11 (RC1) is available

+1 * Signature: ok * Checksum : ok * Rat check (1.8.0_301): ok - mvn clean apache-rat:check * Built from source (1.8.0_301): ok - mvn clean install -DskipTests * Unit tests pass (1.8.0_301): failed (TestUsersOperationsWithSecureHadoop.testAuthUtilLogin) - mvn package -P runSmallTests -Dsuref

[jira] [Created] (HBASE-26850) Optimize the implementation of LRUCache in LRUDictionary

tianhang tang created HBASE-26850: - Summary: Optimize the implementation of LRUCache in LRUDictionary Key: HBASE-26850 URL: https://issues.apache.org/jira/browse/HBASE-26850 Project: HBase Is

[DISCUSS] HBASE-26849 NPE caused by WAL Compression and Replication

Hi masters, I have created an issue HBASE-26849 about NPE caused by WAL Compression and Replication. For this problem, I try to reopen a WAL reader when we reset the position to 0 and it looks like it's working well. But it didn't fundamentally

[jira] [Created] (HBASE-26849) NPE caused by WAL Compression and Replication

tianhang tang created HBASE-26849: - Summary: NPE caused by WAL Compression and Replication Key: HBASE-26849 URL: https://issues.apache.org/jira/browse/HBASE-26849 Project: HBase Issue Type: B