Hi, Thank you for picking this up so quickly. I have no objections regarding all the proposed items. @Gyula: Once the bahir contribution is properly reviewed, ping me if you need somebody to merge it.
On Fri, Aug 7, 2020 at 10:43 AM Márton Balassi <[email protected]> wrote: > Hi Robert and Gyula, > > Thanks for reviving this thread. We have the implementation (currently for > 2.2.3) and it is straightforward to contribute it back. Miklos (ccd) has > recently written a readme for said version, he would be interested in > contributing the upgraded connector back. The latest HBase version is > 2.3.0, if we are touching the codebase anyway I would propose to have that. > > If everyone is comfortable with it I would assign [1] to Miklos with > double checking the all functionality that Felipe has proposed is included. > [1] https://issues.apache.org/jira/browse/FLINK-18795 > [2] https://hbase.apache.org/downloads.html > > On Fri, Aug 7, 2020 at 10:13 AM Gyula Fóra <[email protected]> wrote: > >> Hi Robert, >> >> I completely agree with you on the Bahir based approach. >> >> I am happy to help with the contribution on the bahir side, with thorough >> review and testing. >> >> Cheers, >> Gyula >> >> On Fri, 7 Aug 2020 at 09:30, Robert Metzger <[email protected]> wrote: >> >>> It seems that this thead is not on dev@ anymore. Adding it back ... >>> >>> On Fri, Aug 7, 2020 at 9:23 AM Robert Metzger <[email protected]> >>> wrote: >>> >>>> I would like to revive this discussion. There's a new JIRA[1] + PR[2] >>>> for adding HBase 2 support. >>>> >>>> it seems that there is demand for a HBase 2 connector, and consensus to >>>> do it. >>>> >>>> The remaining question in this thread seems to be the "how". I would >>>> propose to go the other way around as Gyula suggested: We move the legacy >>>> connector (1.4x) to bahir and add the new (2.x.x) to Flink. >>>> Why? In the Flink repo, we have a pretty solid testing infra, where we >>>> also run Hbase end to end tests. This will help us to stabilize the new >>>> connector and ensure a good quality. >>>> It also, the perception of what goes into Flink, and what into Bahir is >>>> a bit clearer if we put the stable, up to date stuff into Flink, and >>>> legacy, experimental or unstable connectors into Bahir. >>>> >>>> >>>> Who can take care of this effort? (Decide which Hbase 2 PR to take, >>>> review and contribution to Bahir) >>>> >>>> >>>> [1] https://issues.apache.org/jira/browse/FLINK-18795 >>>> [2] https://github.com/apache/flink/pull/13047 >>>> >>>> On Mon, Jun 22, 2020 at 3:32 PM Gyula Fóra <[email protected]> >>>> wrote: >>>> >>>>> If we were to go the bahir route, I don't see the point in migrating >>>>> the 1.4.x version there since that's already available in Flink. To me >>>>> that >>>>> is almost the same as dropping explicit support for 1.4 and telling users >>>>> to use older connector versions if they wish to keep using it. >>>>> >>>>> If we want to keep 1.4 around for legacy users and slowly deprecate >>>>> that, we can do that inside Flink and only push the 2.4.x version to >>>>> bahir. >>>>> >>>>> What do you think? >>>>> >>>>> Gyula >>>>> >>>>> On Mon, Jun 22, 2020 at 3:16 PM Arvid Heise <[email protected]> >>>>> wrote: >>>>> >>>>>> If we support both HBase 1 and 2, maybe it's a good time to pull them >>>>>> out to Bahir and list them in flink-packages to avoid adding even more >>>>>> modules to Flink core? >>>>>> >>>>>> On Mon, Jun 22, 2020 at 4:05 AM OpenInx <[email protected]> wrote: >>>>>> >>>>>>> Hi >>>>>>> >>>>>>> According to my observation in the hbase community, there are still >>>>>>> lots of hbase users running their production cluster with version 1.x >>>>>>> (1.4x >>>>>>> or 1.5.x). so I'd like to suggest that >>>>>>> supporting both hbase1.x & hbase2.x connector. >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> On Sat, Jun 20, 2020 at 2:41 PM Ming Li <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> +1 to support both HBase 2.x and Hbase 1.4.x, just as what we are >>>>>>>> doing for Kafka. >>>>>>>> >>>>>>>> On Fri, Jun 19, 2020 at 4:02 PM Yu Li <[email protected]> wrote: >>>>>>>> >>>>>>>>> One supplement: >>>>>>>>> >>>>>>>>> I noticed that there are discussions in HBase ML this March about >>>>>>>>> removing stable-1 pointer and got consensus [1], and will follow up in >>>>>>>>> HBase community about why we didn't take real action. However, this >>>>>>>>> doesn't >>>>>>>>> change my previous statement / stand due to the number of 1.x usages >>>>>>>>> in >>>>>>>>> production. >>>>>>>>> >>>>>>>>> Best Regards, >>>>>>>>> Yu >>>>>>>>> >>>>>>>>> [1] >>>>>>>>> http://mail-archives.apache.org/mod_mbox/hbase-dev/202003.mbox/%[email protected]%3E >>>>>>>>> >>>>>>>>> On Fri, 19 Jun 2020 at 15:54, Yu Li <[email protected]> wrote: >>>>>>>>> >>>>>>>>>> +1 on upgrading the HBase version of the connector, and 1.4.3 is >>>>>>>>>> indeed an old version. >>>>>>>>>> >>>>>>>>>> OTOH, AFAIK there're still quite some 1.x HBase clusters in >>>>>>>>>> production. We could also see that the HBase community is still >>>>>>>>>> maintaining >>>>>>>>>> 1.x release lines (with "stable-1 release" point to 1.4.13) [1] >>>>>>>>>> >>>>>>>>>> Please also notice that HBase follows semantic versioning [2] [3] >>>>>>>>>> thus don't promise any kind of compatibility (source/binary/wire, >>>>>>>>>> etc.) >>>>>>>>>> between major versions. So if we only maintain 2.x connector, it >>>>>>>>>> would not >>>>>>>>>> be able to work with 1.x HBase clusters. >>>>>>>>>> >>>>>>>>>> I totally understand the additional efforts of maintaining two >>>>>>>>>> modules, but since we're also reserving multiple versions for kafka >>>>>>>>>> connector, meantime considering the current HBase in-production >>>>>>>>>> status, I'd >>>>>>>>>> still suggest to get both 1.4.13 and 2.2.5 supported. >>>>>>>>>> >>>>>>>>>> Best Regards, >>>>>>>>>> Yu >>>>>>>>>> >>>>>>>>>> [1] http://hbase.apache.org/downloads.html >>>>>>>>>> [2] https://hbase.apache.org/book.html#hbase.versioning >>>>>>>>>> [3] https://semver.org/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, 19 Jun 2020 at 14:58, Leonard Xu <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> +1 to support HBase 2.2.x, and +1 to retain HBase 1.4.3 until we >>>>>>>>>>> deprecates finished(maybe one version is enough). >>>>>>>>>>> >>>>>>>>>>> Currently we only support HBase 1.4.3 which is pretty old, and >>>>>>>>>>> I’m making a flink-sql-connector-hbase[1] shaded jar for pure SQL >>>>>>>>>>> user, the >>>>>>>>>>> dependencies is a little more complex. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 在 2020年6月19日，14:20，jackylau <[email protected]> 写道： >>>>>>>>>>> >>>>>>>>>>> + 1 to support HBase 2.x and the hbase 2.x client dependencies >>>>>>>>>>> are simple and clear. Tbe hbase project shades them all >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Best, >>>>>>>>>>> Leonard Xu >>>>>>>>>>> [1] https://github.com/apache/flink/pull/12687 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Best Regards >>>>>>>> Michael Li >>>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Arvid Heise | Senior Java Developer >>>>>> >>>>>> <https://www.ververica.com/> >>>>>> >>>>>> Follow us @VervericaData >>>>>> >>>>>> -- >>>>>> >>>>>> Join Flink Forward <https://flink-forward.org/> - The Apache Flink >>>>>> Conference >>>>>> >>>>>> Stream Processing | Event Driven | Real Time >>>>>> >>>>>> -- >>>>>> >>>>>> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany >>>>>> <https://www.google.com/maps/search/Invalidenstrasse+115,+10115+Berlin,+Germany?entry=gmail&source=g> >>>>>> >>>>>> -- >>>>>> Ververica GmbH >>>>>> Registered at Amtsgericht Charlottenburg: HRB 158244 B >>>>>> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, >>>>>> Ji (Toni) Cheng >>>>>> >>>>>
