Re: [DISCUSS] Kubernetes Orchestration for ZK, HDFS, and HBase

2023-03-17 Thread Andrew Purtell
We have also completed a migration to kubernetes (more specifically, EKS) at $dayjob. Our basic deployment stack is helm, terraform, and spinnaker, so we took a different approach in implementation. I think we would be happy to review and offer advice/opinion/suggestion on points of commonality,

Re: [DISCUSS] Kubernetes Orchestration for ZK, HDFS, and HBase

2023-03-17 Thread Tak Lon (Stephen) Wu
Hi Nick, Other than your concerns on the Apache infrastructure, our end are interests and discussed internally about how HBase could be deployed in K8s, but would it be possible to share any architecture diagram or design documentation? Although moving ZK, HDFS, HBase sounds like as simple as

[jira] [Created] (HBASE-27731) Upgrade commons-validator to version 1.7

2023-03-17 Thread Wes Schuitema (Jira)
Wes Schuitema created HBASE-27731: - Summary: Upgrade commons-validator to version 1.7 Key: HBASE-27731 URL: https://issues.apache.org/jira/browse/HBASE-27731 Project: HBase Issue Type: Task

Re: [DISCUSS] Remove checkRefCnt for on-heap ByteBuffs?

2023-03-17 Thread Bryan Beaudreault
Much thanks to Duo for his review, this will be merged shortly to active branches. In the PR we brainstormed ways to also optimize *off-heap* ByteBuffs. That discussion is continuing in https://issues.apache.org/jira/browse/HBASE-27730 if interested. On Tue, Mar 14, 2023 at 8:45 PM Bryan

Re: [ANNOUNCE] New HBase committer Tianhang Tang(唐天航)

2023-03-17 Thread Nihal Jain
Congratulations Tianhang  On Fri, 17 Mar, 2023, 13:09 Xiaolin Ha, wrote: > Congratulations! > > 张铎(Duo Zhang) 于2023年3月16日周四 12:02写道: > > > On behalf of the Apache HBase PMC, I am pleased to announce that Tianhang > > Tang(thangTang) > > has accepted the PMC's invitation to become a committer

[jira] [Created] (HBASE-27730) Optimize reference counting in off-heap ByteBuff

2023-03-17 Thread Bryan Beaudreault (Jira)
Bryan Beaudreault created HBASE-27730: - Summary: Optimize reference counting in off-heap ByteBuff Key: HBASE-27730 URL: https://issues.apache.org/jira/browse/HBASE-27730 Project: HBase

Re: [DISCUSS] Kubernetes Orchestration for ZK, HDFS, and HBase

2023-03-17 Thread Nick Dimiduk
Thank you both for your comments. I'm happy to hear that there is some interest in this pursuit. Before we get too deep into details of the implementation, I'm concerned that we have support from ASF Infra. Can we install KIND or MiniKube on Jenkins worker hosts? Can we get a full cluster

Re: [DISCUSS] How to align hbase-default.xml and hbase-default.adoc

2023-03-17 Thread Nick Dimiduk
I agree that this is an important detail to keep consistent. It's further complicated because things change between release versions, and the book usually only speaks to the current master version. I don't know how the tools work, but I believe that we generate portions of the book from the

[jira] [Created] (HBASE-27729) Missed one parameter when logging exception in StoreFileListFile

2023-03-17 Thread Duo Zhang (Jira)
Duo Zhang created HBASE-27729: - Summary: Missed one parameter when logging exception in StoreFileListFile Key: HBASE-27729 URL: https://issues.apache.org/jira/browse/HBASE-27729 Project: HBase

Re: [ANNOUNCE] New HBase committer Tianhang Tang(唐天航)

2023-03-17 Thread Xiaolin Ha
Congratulations! 张铎(Duo Zhang) 于2023年3月16日周四 12:02写道: > On behalf of the Apache HBase PMC, I am pleased to announce that Tianhang > Tang(thangTang) > has accepted the PMC's invitation to become a committer on the project. We > appreciate all > of Tianhang's generous contributions thus far and

Re: [ANNOUNCE] New HBase committer Tianhang Tang(唐天航)

2023-03-17 Thread Hui Fei
Congratulations, Tianhang! 张铎(Duo Zhang) 于2023年3月16日周四 12:02写道: > On behalf of the Apache HBase PMC, I am pleased to announce that Tianhang > Tang(thangTang) > has accepted the PMC's invitation to become a committer on the project. We > appreciate all > of Tianhang's generous contributions thus

Re: [DISCUSS] Add Ozone as dependency to hbase-asyncfs ?

2023-03-17 Thread Zach York
With S3, we use reflection when loading the filesystem. As long as it is on the CP, we're okay. However, I think the difference here is the APIs that Stephen wants to use is not in the hadoop filesystem API/class, but instead in DFS I believe. Others in this thread have mentioned ways to work

Re: [DISCUSS] Add Ozone as dependency to hbase-asyncfs ?

2023-03-17 Thread Duo Zhang
I just mean how do we manage the dependencies. We do not ship hbase with aws-sdk, but I believe users can put the related jars under lib and then use S3AFileSystem. I think we can support ozone in the same way? Viraj Jasani 于2023年3月17日周五 13:33写道: > WAL is still kept on hdfs only, even when