Hi Robert and Gyula, Thanks for reviving this thread. We have the implementation (currently for 2.2.3) and it is straightforward to contribute it back. Miklos (ccd) has recently written a readme for said version, he would be interested in contributing the upgraded connector back. The latest HBase version is 2.3.0, if we are touching the codebase anyway I would propose to have that.
If everyone is comfortable with it I would assign [1] to Miklos with double checking the all functionality that Felipe has proposed is included. [1] https://issues.apache.org/jira/browse/FLINK-18795 [2] https://hbase.apache.org/downloads.html On Fri, Aug 7, 2020 at 10:13 AM Gyula Fóra <[email protected]> wrote: > Hi Robert, > > I completely agree with you on the Bahir based approach. > > I am happy to help with the contribution on the bahir side, with thorough > review and testing. > > Cheers, > Gyula > > On Fri, 7 Aug 2020 at 09:30, Robert Metzger <[email protected]> wrote: > >> It seems that this thead is not on dev@ anymore. Adding it back ... >> >> On Fri, Aug 7, 2020 at 9:23 AM Robert Metzger <[email protected]> >> wrote: >> >>> I would like to revive this discussion. There's a new JIRA[1] + PR[2] >>> for adding HBase 2 support. >>> >>> it seems that there is demand for a HBase 2 connector, and consensus to >>> do it. >>> >>> The remaining question in this thread seems to be the "how". I would >>> propose to go the other way around as Gyula suggested: We move the legacy >>> connector (1.4x) to bahir and add the new (2.x.x) to Flink. >>> Why? In the Flink repo, we have a pretty solid testing infra, where we >>> also run Hbase end to end tests. This will help us to stabilize the new >>> connector and ensure a good quality. >>> It also, the perception of what goes into Flink, and what into Bahir is >>> a bit clearer if we put the stable, up to date stuff into Flink, and >>> legacy, experimental or unstable connectors into Bahir. >>> >>> >>> Who can take care of this effort? (Decide which Hbase 2 PR to take, >>> review and contribution to Bahir) >>> >>> >>> [1] https://issues.apache.org/jira/browse/FLINK-18795 >>> [2] https://github.com/apache/flink/pull/13047 >>> >>> On Mon, Jun 22, 2020 at 3:32 PM Gyula Fóra <[email protected]> wrote: >>> >>>> If we were to go the bahir route, I don't see the point in migrating >>>> the 1.4.x version there since that's already available in Flink. To me that >>>> is almost the same as dropping explicit support for 1.4 and telling users >>>> to use older connector versions if they wish to keep using it. >>>> >>>> If we want to keep 1.4 around for legacy users and slowly deprecate >>>> that, we can do that inside Flink and only push the 2.4.x version to bahir. >>>> >>>> What do you think? >>>> >>>> Gyula >>>> >>>> On Mon, Jun 22, 2020 at 3:16 PM Arvid Heise <[email protected]> >>>> wrote: >>>> >>>>> If we support both HBase 1 and 2, maybe it's a good time to pull them >>>>> out to Bahir and list them in flink-packages to avoid adding even more >>>>> modules to Flink core? >>>>> >>>>> On Mon, Jun 22, 2020 at 4:05 AM OpenInx <[email protected]> wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> According to my observation in the hbase community, there are still >>>>>> lots of hbase users running their production cluster with version 1.x >>>>>> (1.4x >>>>>> or 1.5.x). so I'd like to suggest that >>>>>> supporting both hbase1.x & hbase2.x connector. >>>>>> >>>>>> Thanks. >>>>>> >>>>>> On Sat, Jun 20, 2020 at 2:41 PM Ming Li <[email protected]> wrote: >>>>>> >>>>>>> +1 to support both HBase 2.x and Hbase 1.4.x, just as what we are >>>>>>> doing for Kafka. >>>>>>> >>>>>>> On Fri, Jun 19, 2020 at 4:02 PM Yu Li <[email protected]> wrote: >>>>>>> >>>>>>>> One supplement: >>>>>>>> >>>>>>>> I noticed that there are discussions in HBase ML this March about >>>>>>>> removing stable-1 pointer and got consensus [1], and will follow up in >>>>>>>> HBase community about why we didn't take real action. However, this >>>>>>>> doesn't >>>>>>>> change my previous statement / stand due to the number of 1.x usages in >>>>>>>> production. >>>>>>>> >>>>>>>> Best Regards, >>>>>>>> Yu >>>>>>>> >>>>>>>> [1] >>>>>>>> http://mail-archives.apache.org/mod_mbox/hbase-dev/202003.mbox/%[email protected]%3E >>>>>>>> >>>>>>>> On Fri, 19 Jun 2020 at 15:54, Yu Li <[email protected]> wrote: >>>>>>>> >>>>>>>>> +1 on upgrading the HBase version of the connector, and 1.4.3 is >>>>>>>>> indeed an old version. >>>>>>>>> >>>>>>>>> OTOH, AFAIK there're still quite some 1.x HBase clusters in >>>>>>>>> production. We could also see that the HBase community is still >>>>>>>>> maintaining >>>>>>>>> 1.x release lines (with "stable-1 release" point to 1.4.13) [1] >>>>>>>>> >>>>>>>>> Please also notice that HBase follows semantic versioning [2] [3] >>>>>>>>> thus don't promise any kind of compatibility (source/binary/wire, >>>>>>>>> etc.) >>>>>>>>> between major versions. So if we only maintain 2.x connector, it >>>>>>>>> would not >>>>>>>>> be able to work with 1.x HBase clusters. >>>>>>>>> >>>>>>>>> I totally understand the additional efforts of maintaining two >>>>>>>>> modules, but since we're also reserving multiple versions for kafka >>>>>>>>> connector, meantime considering the current HBase in-production >>>>>>>>> status, I'd >>>>>>>>> still suggest to get both 1.4.13 and 2.2.5 supported. >>>>>>>>> >>>>>>>>> Best Regards, >>>>>>>>> Yu >>>>>>>>> >>>>>>>>> [1] http://hbase.apache.org/downloads.html >>>>>>>>> [2] https://hbase.apache.org/book.html#hbase.versioning >>>>>>>>> [3] https://semver.org/ >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, 19 Jun 2020 at 14:58, Leonard Xu <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> +1 to support HBase 2.2.x, and +1 to retain HBase 1.4.3 until we >>>>>>>>>> deprecates finished(maybe one version is enough). >>>>>>>>>> >>>>>>>>>> Currently we only support HBase 1.4.3 which is pretty old, and >>>>>>>>>> I’m making a flink-sql-connector-hbase[1] shaded jar for pure SQL >>>>>>>>>> user, the >>>>>>>>>> dependencies is a little more complex. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 在 2020年6月19日，14:20，jackylau <[email protected]> 写道： >>>>>>>>>> >>>>>>>>>> + 1 to support HBase 2.x and the hbase 2.x client dependencies >>>>>>>>>> are simple and clear. Tbe hbase project shades them all >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> Leonard Xu >>>>>>>>>> [1] https://github.com/apache/flink/pull/12687 >>>>>>>>>> >>>>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Best Regards >>>>>>> Michael Li >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> >>>>> Arvid Heise | Senior Java Developer >>>>> >>>>> <https://www.ververica.com/> >>>>> >>>>> Follow us @VervericaData >>>>> >>>>> -- >>>>> >>>>> Join Flink Forward <https://flink-forward.org/> - The Apache Flink >>>>> Conference >>>>> >>>>> Stream Processing | Event Driven | Real Time >>>>> >>>>> -- >>>>> >>>>> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany >>>>> <https://www.google.com/maps/search/Invalidenstrasse+115,+10115+Berlin,+Germany?entry=gmail&source=g> >>>>> >>>>> -- >>>>> Ververica GmbH >>>>> Registered at Amtsgericht Charlottenburg: HRB 158244 B >>>>> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, >>>>> Ji (Toni) Cheng >>>>> >>>>
