Hi Robert, I completely agree with you on the Bahir based approach.
I am happy to help with the contribution on the bahir side, with thorough review and testing. Cheers, Gyula On Fri, 7 Aug 2020 at 09:30, Robert Metzger <rmetz...@apache.org> wrote: > It seems that this thead is not on dev@ anymore. Adding it back ... > > On Fri, Aug 7, 2020 at 9:23 AM Robert Metzger <rmetz...@apache.org> wrote: > >> I would like to revive this discussion. There's a new JIRA[1] + PR[2] for >> adding HBase 2 support. >> >> it seems that there is demand for a HBase 2 connector, and consensus to >> do it. >> >> The remaining question in this thread seems to be the "how". I would >> propose to go the other way around as Gyula suggested: We move the legacy >> connector (1.4x) to bahir and add the new (2.x.x) to Flink. >> Why? In the Flink repo, we have a pretty solid testing infra, where we >> also run Hbase end to end tests. This will help us to stabilize the new >> connector and ensure a good quality. >> It also, the perception of what goes into Flink, and what into Bahir is a >> bit clearer if we put the stable, up to date stuff into Flink, and legacy, >> experimental or unstable connectors into Bahir. >> >> >> Who can take care of this effort? (Decide which Hbase 2 PR to take, >> review and contribution to Bahir) >> >> >> [1] https://issues.apache.org/jira/browse/FLINK-18795 >> [2] https://github.com/apache/flink/pull/13047 >> >> On Mon, Jun 22, 2020 at 3:32 PM Gyula Fóra <gyula.f...@gmail.com> wrote: >> >>> If we were to go the bahir route, I don't see the point in migrating the >>> 1.4.x version there since that's already available in Flink. To me that is >>> almost the same as dropping explicit support for 1.4 and telling users to >>> use older connector versions if they wish to keep using it. >>> >>> If we want to keep 1.4 around for legacy users and slowly deprecate >>> that, we can do that inside Flink and only push the 2.4.x version to bahir. >>> >>> What do you think? >>> >>> Gyula >>> >>> On Mon, Jun 22, 2020 at 3:16 PM Arvid Heise <ar...@ververica.com> wrote: >>> >>>> If we support both HBase 1 and 2, maybe it's a good time to pull them >>>> out to Bahir and list them in flink-packages to avoid adding even more >>>> modules to Flink core? >>>> >>>> On Mon, Jun 22, 2020 at 4:05 AM OpenInx <open...@gmail.com> wrote: >>>> >>>>> Hi >>>>> >>>>> According to my observation in the hbase community, there are still >>>>> lots of hbase users running their production cluster with version 1.x >>>>> (1.4x >>>>> or 1.5.x). so I'd like to suggest that >>>>> supporting both hbase1.x & hbase2.x connector. >>>>> >>>>> Thanks. >>>>> >>>>> On Sat, Jun 20, 2020 at 2:41 PM Ming Li <brighty...@gmail.com> wrote: >>>>> >>>>>> +1 to support both HBase 2.x and Hbase 1.4.x, just as what we are >>>>>> doing for Kafka. >>>>>> >>>>>> On Fri, Jun 19, 2020 at 4:02 PM Yu Li <car...@gmail.com> wrote: >>>>>> >>>>>>> One supplement: >>>>>>> >>>>>>> I noticed that there are discussions in HBase ML this March about >>>>>>> removing stable-1 pointer and got consensus [1], and will follow up in >>>>>>> HBase community about why we didn't take real action. However, this >>>>>>> doesn't >>>>>>> change my previous statement / stand due to the number of 1.x usages in >>>>>>> production. >>>>>>> >>>>>>> Best Regards, >>>>>>> Yu >>>>>>> >>>>>>> [1] >>>>>>> http://mail-archives.apache.org/mod_mbox/hbase-dev/202003.mbox/%3c30180be2-bd93-d414-a158-16c9c8d01...@apache.org%3E >>>>>>> >>>>>>> On Fri, 19 Jun 2020 at 15:54, Yu Li <car...@gmail.com> wrote: >>>>>>> >>>>>>>> +1 on upgrading the HBase version of the connector, and 1.4.3 is >>>>>>>> indeed an old version. >>>>>>>> >>>>>>>> OTOH, AFAIK there're still quite some 1.x HBase clusters in >>>>>>>> production. We could also see that the HBase community is still >>>>>>>> maintaining >>>>>>>> 1.x release lines (with "stable-1 release" point to 1.4.13) [1] >>>>>>>> >>>>>>>> Please also notice that HBase follows semantic versioning [2] [3] >>>>>>>> thus don't promise any kind of compatibility (source/binary/wire, etc.) >>>>>>>> between major versions. So if we only maintain 2.x connector, it would >>>>>>>> not >>>>>>>> be able to work with 1.x HBase clusters. >>>>>>>> >>>>>>>> I totally understand the additional efforts of maintaining two >>>>>>>> modules, but since we're also reserving multiple versions for kafka >>>>>>>> connector, meantime considering the current HBase in-production >>>>>>>> status, I'd >>>>>>>> still suggest to get both 1.4.13 and 2.2.5 supported. >>>>>>>> >>>>>>>> Best Regards, >>>>>>>> Yu >>>>>>>> >>>>>>>> [1] http://hbase.apache.org/downloads.html >>>>>>>> [2] https://hbase.apache.org/book.html#hbase.versioning >>>>>>>> [3] https://semver.org/ >>>>>>>> >>>>>>>> >>>>>>>> On Fri, 19 Jun 2020 at 14:58, Leonard Xu <xbjt...@gmail.com> wrote: >>>>>>>> >>>>>>>>> +1 to support HBase 2.2.x, and +1 to retain HBase 1.4.3 until we >>>>>>>>> deprecates finished(maybe one version is enough). >>>>>>>>> >>>>>>>>> Currently we only support HBase 1.4.3 which is pretty old, and I’m >>>>>>>>> making a flink-sql-connector-hbase[1] shaded jar for pure SQL user, >>>>>>>>> the >>>>>>>>> dependencies is a little more complex. >>>>>>>>> >>>>>>>>> >>>>>>>>> 在 2020年6月19日，14:20，jackylau <liuyon...@gmail.com> 写道： >>>>>>>>> >>>>>>>>> + 1 to support HBase 2.x and the hbase 2.x client dependencies are >>>>>>>>> simple and clear. Tbe hbase project shades them all >>>>>>>>> >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Leonard Xu >>>>>>>>> [1] https://github.com/apache/flink/pull/12687 >>>>>>>>> >>>>>>>>> >>>>>> >>>>>> -- >>>>>> Best Regards >>>>>> Michael Li >>>>>> >>>>> >>>> >>>> -- >>>> >>>> Arvid Heise | Senior Java Developer >>>> >>>> <https://www.ververica.com/> >>>> >>>> Follow us @VervericaData >>>> >>>> -- >>>> >>>> Join Flink Forward <https://flink-forward.org/> - The Apache Flink >>>> Conference >>>> >>>> Stream Processing | Event Driven | Real Time >>>> >>>> -- >>>> >>>> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany >>>> <https://www.google.com/maps/search/Invalidenstrasse+115,+10115+Berlin,+Germany?entry=gmail&source=g> >>>> >>>> -- >>>> Ververica GmbH >>>> Registered at Amtsgericht Charlottenburg: HRB 158244 B >>>> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji >>>> (Toni) Cheng >>>> >>>
