Re: [DISCUSS] Contribution of a thrift2 python api

[Duo Zhang]

One of the difficulties of moving hbase-thrift and hbase-rest out is
because we make use of hbase-http in these two modules, at least for
setting up the status servlet...

Sean Busbey  于2021年7月16日周五 下午11:01写道：

> maybe a good fit for the hbase-connectors repo? I know we've talked a
> few times about moving the thrift server out there. if we did both
> then the compatibility question becomes just the standard
> client/server compatibility provided the thrift server only uses our
> public java client API.
>
> On Thu, Jul 15, 2021 at 10:21 PM Yutong Xiao  wrote:
> >
> > btw for point 2, if allowed I can do that.
> > And for point 3.2 it is only a personal idea, the final decision should
> be
> > made by the community.
> > Besides, many of my python user colleagues started using this library.
> > I think many python users have the demand of a good HBase python client.
> >
> > Yutong Xiao  于2021年7月16日周五 上午11:07写道：
> >
> > > 1. The license is no problem.
> > > 2. This should see if any committer or PMC has interests to do that.
> > > 3. I can be responsible for those documents. About 3.2, as thbase has
> been
> > > uploaded to Pypi, I think it would be better if it is a new, separate
> repo.
> > >
> > > Wei-Chiu Chuang  于2021年7月6日周二 上午10:22写道：
> > >
> > >> Hi
> > >> thanks for your interest in contributing the python api to the HBase
> > >> project.
> > >>
> > >> I quickly check and it doesn't look like there's another active python
> > >> HBase thrift client project at this point.
> > >> I don't have a demand to use a python thrift hbase client library. If
> > >> there
> > >> are people who will benefit from this library, then it's a good idea
> to
> > >> make sure the library is well maintained, by having it become part of
> the
> > >> Apache HBase project and that more developers can contribute to it.
> > >>
> > >> As a hobbyist Python developer I can help review/commit the patch.
> > >>
> > >> My two cents:
> > >> (1) license: the code is ASL 2.0 so it's compatible. The text
> "Copyright
> > >> 2021 Yutong Sean" would need to be removed.
> > >> (2) Apache Infra does not manage PyPi. So we (the Apache HBase project
> > >> committers/PMC) will have to do that.
> > >> I suspect we will have to replicate this PyPi project and add the
> > >> interested HBase PMCs who's willing to do the release work.
> > >> (3) compatibility matrix: we need to document what versions of HBase
> > >> server
> > >> is supported.
> > >> (3) code:
> > >> (3.1) You will need a requirements.txt and preferably specify the
> versions
> > >> of the dependencies.
> > >> (3.2) If the community accepts it, should it be part of the HBase main
> > >> repo, or a new, separate repo?
> > >>
> > >>
> > >>
> > >> On Mon, Jul 5, 2021 at 7:12 PM Yutong Xiao 
> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > I used to have a demand to deploy hbase thrift2 service for python
> > >> users.
> > >> > So that I developed a python clients API supporting python 2.7 and
> 3.x
> > >> for
> > >> > hbase thrift2, named thbase  .
> > >> Besides
> > >> > that, I also added some features to current thrift2 service
> (HBASE-26025
> > >> > 
> and
> > >> > HBASE-26037
> > >> > ).
> I
> > >> > deployed them in the prod environment of my company and are
> compatible
> > >> with
> > >> > thbase and I will keep maintaining this python API and add new
> features.
> > >> >   I am glad to contribute thbase to the community, but I am not
> sure if
> > >> it
> > >> > is possible that such a client could be contributed to the
> community. So
> > >> > that I would like to get some advice about this.
> > >> >
> > >> > Thanks,
> > >> > Yutong Sean
> > >> >
> > >>
> > >
>

[DISCUSS] Remove FirstKeyValueMatchingQualifiersFilter

[Huang Zhuoyue]

Hi, 


As https://issues.apache.org/jira/browse/HBASE-13347 describes, the 
FirstKeyValueMatchingQualifiersFilter was deprecated in 2.0 and will be removed 
in 3.0. Now we need to remove it in 3.0 but there are proto declareation of 
FirstKeyValueMatchingQualifiersFilter in Filter.proto. 


If the proto declaration of FirstKeyValueMatchingQualifiersFilter is removed, 
serialization problems and unknown errors may occur when the old version of 
hbase-client accesses the new version of the server, which may puzzle users.


Should we remove the proto declaration directly? Or throw RuntimeException when 
the user uses FirstKeyValueMatchingQualifiersFilter? Hope to get your 
suggestions.


Thanks,
Zhuoyue Huang

Re: Time for 2.4.5

[Andrew Purtell]

Yes, let’s wait. Thank you for the notice. 

> On Jul 16, 2021, at 11:01 AM, Huaxiang Sun  wrote:
> 
> Sorry, typo. s/we run into this core dump quite often as wekk/we run into
> this core dump quite often as well.
> 
>> On Fri, Jul 16, 2021 at 10:58 AM Huaxiang Sun  wrote:
>> 
>> Hi Andrew,
>> 
>>   Do we want to wait a few days for HBASE-24984
>> ? Several users (we
>> run into this core dump quite often as wekk) reported core dump and a fix
>> is ready.
>> 
>>   Thanks,
>>   Huaxiang
>> 
>> On Sat, Jul 10, 2021 at 7:35 PM Andrew Purtell 
>> wrote:
>> 
>>> It’s about time for 2.4.5.
>>> 
>>> If there is pending work you’d like to see go out in the release, please
>>> commit it in the next couple of days. Any potential blockers, please raise
>>> them here in this thread.
>>> 
>>> Planning to have an RC by the end of the week.
>>> 
>>> 

Re: [ANNOUNCE] New HBase committer Baiqiang Zhao

[Huaxiang Sun]

Congratulations Baiqiang!

On Tue, Jul 13, 2021 at 11:10 AM Zach York 
wrote:

> Congratulations and welcome Baiqiang!
>
> On Mon, Jul 12, 2021 at 4:59 AM Baiqiang Zhao  wrote:
>
> > Thank you all for your warm welcome!
> >
> > Peter Somogyi  于2021年7月12日周一 下午7:46写道：
> >
> > > Congratulations!
> > >
> > > On Mon, Jul 12, 2021 at 12:53 PM Yu Li  wrote:
> > >
> > > > Congrats and welcome, Baiqiang.
> > > >
> > > > Best Regards,
> > > > Yu
> > > >
> > > >
> > > > On Mon, 12 Jul 2021 at 16:56, 哈晓琳  wrote:
> > > >
> > > > > Congratulations Baiqiang!
> > > > >
> > > > > Balazs Meszaros 
> 于2021年7月12日周一
> > > > > 下午4:35写道：
> > > > >
> > > > > > Congratulations Baiqiang!
> > > > > >
> > > > > > On Mon, Jul 12, 2021 at 9:22 AM Pankaj Kumar <
> > pankajku...@apache.org
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Congratulations Baiqiang.!!
> > > > > > >
> > > > > > > Regards,
> > > > > > > Pankaj
> > > > > > >
> > > > > > > On Sun, Jul 11, 2021, 12:48 AM Nick Dimiduk <
> ndimi...@apache.org
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Hi everyone,
> > > > > > > >
> > > > > > > > On behalf of the Apache HBase PMC I am pleased to announce
> that
> > > > > > Baiqiang
> > > > > > > > Zhao has accepted the PMC's invitation to become a committer
> on
> > > the
> > > > > > > > project!
> > > > > > > >
> > > > > > > > We appreciate all of the great contributions Baiqiang has
> made
> > to
> > > > > > > > the community thus far and we look forward to his continued
> > > > > > involvement.
> > > > > > > >
> > > > > > > > Allow me to be the first to congratulate Baiqiang on his new
> > > role!
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Nick
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Time for 2.4.5

[Huaxiang Sun]

Sorry, typo. s/we run into this core dump quite often as wekk/we run into
this core dump quite often as well.

On Fri, Jul 16, 2021 at 10:58 AM Huaxiang Sun  wrote:

> Hi Andrew,
>
>Do we want to wait a few days for HBASE-24984
> ? Several users (we
> run into this core dump quite often as wekk) reported core dump and a fix
> is ready.
>
>Thanks,
>Huaxiang
>
> On Sat, Jul 10, 2021 at 7:35 PM Andrew Purtell 
> wrote:
>
>> It’s about time for 2.4.5.
>>
>> If there is pending work you’d like to see go out in the release, please
>> commit it in the next couple of days. Any potential blockers, please raise
>> them here in this thread.
>>
>> Planning to have an RC by the end of the week.
>>
>>

Re: Time for 2.4.5

[Huaxiang Sun]

Hi Andrew,

   Do we want to wait a few days for HBASE-24984
? Several users (we run
into this core dump quite often as wekk) reported core dump and a fix is
ready.

   Thanks,
   Huaxiang

On Sat, Jul 10, 2021 at 7:35 PM Andrew Purtell 
wrote:

> It’s about time for 2.4.5.
>
> If there is pending work you’d like to see go out in the release, please
> commit it in the next couple of days. Any potential blockers, please raise
> them here in this thread.
>
> Planning to have an RC by the end of the week.
>
>

[jira] [Resolved] (HBASE-25739) TableSkewCostFunction need to use aggregated deviation

[Michael Stack (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Stack resolved HBASE-25739.
---
Hadoop Flags: Reviewed
  Resolution: Fixed

Resovling after merging PRs for branch-2.3+. Thanks for the patch 
[~clarax98007]  (and reviews [~busbey] and [~zhangduo] )

> TableSkewCostFunction need to use aggregated deviation
> --
>
> Key: HBASE-25739
> URL: https://issues.apache.org/jira/browse/HBASE-25739
> Project: HBase
>  Issue Type: Sub-task
>  Components: Balancer, master
>Reporter: Clara Xiong
>Assignee: Clara Xiong
>Priority: Major
> Fix For: 2.5.0, 2.3.6, 3.0.0-alpha-2, 2.4.5
>
> Attachments: 
> TEST-org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerBalanceCluster.xml,
>  
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerBalanceCluster.txt
>
>
> TableSkewCostFunction uses the sum of the max deviation region per server for 
> all tables as the measure of unevenness. It doesn't work in a very common 
> scenario in operations. Say we have 100 regions on 50 nodes, two on each. We 
> add 50 new nodes and they have 0 each. The max deviation from the mean is 1, 
> compared to 99 in the worst case scenario of 100 regions on a single server. 
> The normalized cost is 1/99 = 0.011 < default threshold of 0.05. Balancer 
> wouldn't move.  The proposal is to use aggregated deviation of the count per 
> region server to detect this scenario, generating a cost of 100/198 = 0.5 in 
> this case.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[VOTE] The Release Candidate for HBase 1.7.1 (RC0) is available for download

[Bharath Vissapragada]

Hi everyone,

The first release candidate (RC0) for HBase 1.7.1 is available for download:
https://dist.apache.org/repos/dist/dev/hbase/1.7.1RC0/

Release tag:
https://github.com/apache/hbase/tree/1.7.1RC0

Maven artifacts are available in a staging repository at:
https://repository.apache.org/content/repositories/orgapachehbase-1458/

A detailed source and binary compatibility report for this release is at:
https://dist.apache.org/repos/dist/dev/hbase/1.7.1RC0/api_compare_1.7.0_to_1.7.1-RC0.html

Release notes can be found at:

https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12350241=html=12310753=Create_token=A5KQ-2QAV-T4JA-FDED_2640918bc7c9809c3a16f01bd099bb435d5f87ff_lin

Current flaky test report is at:
https://ci-hadoop.apache.org/job/HBase/job/HBase-Find-Flaky-Tests/job/branch-1/Flaky_20Test_20Report/

All the artifacts were signed using my key DC190FA051EE6466 at
https://dist.apache.org/repos/dist/release/hbase/KEYS

Note: Trigger for this release is HBASE-26021
 that addresses an
incompatible table serialization added to 1.7.0. Once the vote on an 1.7.1
RC passes, 1.7.0 will be withdrawn in favor of 1.7.1+. Fixes in this jira
(revert of incompatible serialization) are reflected in the compat-check
report API changes linked above.

Please try out this candidate and vote on it. The vote remains for at least
72 working hours. *Since we are approaching a weekend here in the US (and
is already a weekend on the other side of the globe), I'll check back mid
of next week.*

[ ] +1 Release this package as Apache HBase 1.7.1
[ ] -1 Do not release this package because ...

Thanks,
Bharath.

Re: [DISCUSS] Contribution of a thrift2 python api

[Sean Busbey]

maybe a good fit for the hbase-connectors repo? I know we've talked a
few times about moving the thrift server out there. if we did both
then the compatibility question becomes just the standard
client/server compatibility provided the thrift server only uses our
public java client API.

On Thu, Jul 15, 2021 at 10:21 PM Yutong Xiao  wrote:
>
> btw for point 2, if allowed I can do that.
> And for point 3.2 it is only a personal idea, the final decision should be
> made by the community.
> Besides, many of my python user colleagues started using this library.
> I think many python users have the demand of a good HBase python client.
>
> Yutong Xiao  于2021年7月16日周五 上午11:07写道：
>
> > 1. The license is no problem.
> > 2. This should see if any committer or PMC has interests to do that.
> > 3. I can be responsible for those documents. About 3.2, as thbase has been
> > uploaded to Pypi, I think it would be better if it is a new, separate repo.
> >
> > Wei-Chiu Chuang  于2021年7月6日周二 上午10:22写道：
> >
> >> Hi
> >> thanks for your interest in contributing the python api to the HBase
> >> project.
> >>
> >> I quickly check and it doesn't look like there's another active python
> >> HBase thrift client project at this point.
> >> I don't have a demand to use a python thrift hbase client library. If
> >> there
> >> are people who will benefit from this library, then it's a good idea to
> >> make sure the library is well maintained, by having it become part of the
> >> Apache HBase project and that more developers can contribute to it.
> >>
> >> As a hobbyist Python developer I can help review/commit the patch.
> >>
> >> My two cents:
> >> (1) license: the code is ASL 2.0 so it's compatible. The text "Copyright
> >> 2021 Yutong Sean" would need to be removed.
> >> (2) Apache Infra does not manage PyPi. So we (the Apache HBase project
> >> committers/PMC) will have to do that.
> >> I suspect we will have to replicate this PyPi project and add the
> >> interested HBase PMCs who's willing to do the release work.
> >> (3) compatibility matrix: we need to document what versions of HBase
> >> server
> >> is supported.
> >> (3) code:
> >> (3.1) You will need a requirements.txt and preferably specify the versions
> >> of the dependencies.
> >> (3.2) If the community accepts it, should it be part of the HBase main
> >> repo, or a new, separate repo?
> >>
> >>
> >>
> >> On Mon, Jul 5, 2021 at 7:12 PM Yutong Xiao  wrote:
> >>
> >> > Hi,
> >> >
> >> > I used to have a demand to deploy hbase thrift2 service for python
> >> users.
> >> > So that I developed a python clients API supporting python 2.7 and 3.x
> >> for
> >> > hbase thrift2, named thbase  .
> >> Besides
> >> > that, I also added some features to current thrift2 service (HBASE-26025
> >> >  and
> >> > HBASE-26037
> >> > ). I
> >> > deployed them in the prod environment of my company and are compatible
> >> with
> >> > thbase and I will keep maintaining this python API and add new features.
> >> >   I am glad to contribute thbase to the community, but I am not sure if
> >> it
> >> > is possible that such a client could be contributed to the community. So
> >> > that I would like to get some advice about this.
> >> >
> >> > Thanks,
> >> > Yutong Sean
> >> >
> >>
> >

[jira] [Resolved] (HBASE-26090) Remove the deprecated methods in Scan which should be removed in 3.0.0

[Duo Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang resolved HBASE-26090.
---
Hadoop Flags: Reviewed
  Resolution: Fixed

Merged to master.

Thanks all for reviewing.

> Remove the deprecated methods in Scan which should be removed in 3.0.0
> --
>
> Key: HBASE-26090
> URL: https://issues.apache.org/jira/browse/HBASE-26090
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-2
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-26094) In branch-1 L2 BC should not be the victimhandler of L1 BC when using combined BC

[Yutong Xiao (Jira)]

Yutong Xiao created HBASE-26094:
---

 Summary: In branch-1 L2 BC should not be the victimhandler of L1 
BC when using combined BC
 Key: HBASE-26094
 URL: https://issues.apache.org/jira/browse/HBASE-26094
 Project: HBase
  Issue Type: Bug
  Components: BlockCache
Affects Versions: 1.7.0
Reporter: Yutong Xiao
Assignee: Yutong Xiao


As the logic, the L2 should not be the victimhandler of L1 cache. When using 
InclusiveCombinedBlockCache, we should set L2 as the victimhandler of L1. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)