+1, please do On Mon, Nov 5, 2018 at 9:55 AM Stack <st...@duboce.net> wrote:
> I was going to file an issue w/ INFRA this evening asking that they disable > github comment/commits being added as JIRA comments (See HBASE-21435 for an > innocuous example of this feature in action and HBASE-21430 for an > illustration of how it can quickly turn silly). While I could see > github-comments-appearing-as-JIRA comments working if the changesets were > small, it blows away JIRA if the change has any weight at all. > > I was going to ask that the github/gitbox commentary instead go to issues@ > mailing list (Sean suggestion. Duo and our Peter asked off-list suggested > we not fill JIRA comments). Shout if you are against or have any > preferences. > > Yours, > S > > > > On Fri, Nov 2, 2018 at 11:43 AM Stack <st...@duboce.net> wrote: > > > Related, interesting thread on how various projects are doing the > > github<->asf connection has just started[1]. > > > > A few have a notifications list that gets the github notification emails. > > Email clients make parseable threads of the firehose apparently. We might > > do same. > > > > S > > 1. > > > http://mail-archives.apache.org/mod_mbox/community-dev/201811.mbox/browser > > > > On Fri, Nov 2, 2018 at 10:37 AM Josh Elser <els...@apache.org> wrote: > > > >> > >> > >> On 11/2/18 11:20 AM, Stack wrote: > >> > On Fri, Nov 2, 2018 at 7:30 AM Josh Elser <els...@apache.org> wrote: > >> > > >> >> Nice stuff, Stack! > >> >> > >> >> > >> > Not me. It Mike's work. > >> > > >> > >> Sorry, didn't mean it that way. Was happy to see you pushing the > >> hbase-connectors repo forward :) > >> > >> Obviously kudos to MikeW for the code itself! > >> > >> >> Two quick questions: > >> >> > >> >> First, on provenance: this codebase primarily came from Mike Wingert > on > >> >> https://issues.apache.org/jira/browse/HBASE-15320? Just saw that the > >> >> commit came from your email addr -- wasn't sure if that Mike was > still > >> >> involved (or you took it to completion). > >> >> > >> >> > >> > You talking about the merge done over on the hbase-connectors? > >> > > >> > Looks like I get blamed for the merge -- if I do a show on the merge > >> > commit, it is nothing but the merge note "Merge pull request #3 from > >> > hbasejanitor/master" where hbasejanitor is Mike's handle -- but then > the > >> > merge note is followed by Mikes' work, properly attributed to him. > >> > > >> > I did not pay close attention to this aspect of how git boxing does > it. > >> > Seems fine to me. What you think? > >> > >> Ahh, no, this was just me. I think I only looked at the first commit(s), > >> and not far enough down the list. No concerns. > >> > >> >> Second, I assume this new Git repo had all of the normal email-hooks > >> set > >> >> up. Do you know where they are being sent (dev, commit, or issues)? > I'm > >> >> also assuming that this is a Gitbox repo -- are we OK with > >> pull-requests > >> >> to this repo (as well as operator-tools) but still create a Jira > issue? > >> >> > >> >> > >> > Yep, gitbox. It has whatever infra set it up as. > >> > > >> > Back and forth was dumped into HBASE-21002 (We changed this config on > >> > hbase-operator-tools config to not do this? I should look). > >> > > >> > Regards pull requests, etc., email configs., etc., we are in > >> experimental > >> > mode around all this stuff trying to figure it out so > >> > suggestions/help/exercising the possibilities are all welcome. > >> > > >> > Thanks, > >> > S > >> > >> Grand. Thanks for the reminder. Had a random question in Slack the other > >> day about contributions -- will keep the "pave your own road" mindset in > >> the fore-front :) > >> > >> >> - Josh > >> >> > >> >> On 10/31/18 6:43 PM, Stack wrote: > >> >>> To tie-off this thread, this nice feature was just pushed on > >> >>> hbase-connector. See > >> >>> https://github.com/apache/hbase-connectors/tree/master/kafka for > >> how-to. > >> >>> Review and commentary welcome. > >> >>> > >> >>> Thanks, > >> >>> S > >> >>> > >> >>> On Fri, Aug 3, 2018 at 6:32 AM Hbase Janitor < > hbasejani...@gmail.com> > >> >> wrote: > >> >>> > >> >>>> I opened hbase-21002 to start the scripts and assembly. > >> >>>> > >> >>>> Mike > >> >>>> > >> >>>> On Thu, Aug 2, 2018, 19:29 Stack <st...@duboce.net> wrote: > >> >>>> > >> >>>>> Up in https://issues.apache.org/jira/browse/HBASE-20934 I created > >> an > >> >>>>> hbase-connectors repo. I put some form on it using the v19 patch > >> from > >> >>>>> HBASE-15320 "HBase connector for Kafka Connect". It builds and > tests > >> >>>>> pass. Here are some remaining TODOs: > >> >>>>> > >> >>>>> * Figure how to do start scripts: e.g. we need to start up the > >> kafka > >> >>>>> proxy. It wants some hbase jars, conf dir, and others on the > >> CLASSPATH > >> >>>>> (Depend on an HBASE_HOME and then source bin/hbase?) > >> >>>>> * Can any of the connectors make-do with the shaded client? > >> >>>>> * Make connectors standalone or have them share conf, bin, etc? > >> >>>>> * Need to do an assembly. Not done. > >> >>>>> * Move over REST and thrift next. Mapreduce after? > >> >>>>> > >> >>>>> The poms could do w/ a review. Hacked them over from > >> hbase-thirdparty. > >> >>>>> > >> >>>>> File issues and apply patches up in JIRA if your up for any of the > >> >> above. > >> >>>>> > >> >>>>> Thanks, > >> >>>>> S > >> >>>>> > >> >>>>> On Wed, Jul 25, 2018 at 10:46 PM Stack <st...@duboce.net> wrote: > >> >>>>>> > >> >>>>>> > >> >>>>>> On Tue, Jul 24, 2018 at 10:01 PM Misty Linville < > mi...@apache.org> > >> >>>>> wrote: > >> >>>>>>> > >> >>>>>>> I like the idea of a separate connectors repo/release vehicle, > but > >> >>>> I'm a > >> >>>>>>> little concerned about the need to release all together to > update > >> >> just > >> >>>>> one > >> >>>>>>> of the connectors. How would that work? What kind of > compatibility > >> >>>>>>> guarantees are we signing up for? > >> >>>>>>> > >> >>>>>> > >> >>>>>> I hate responses that begin "Good question" -- so fawning -- but, > >> >> ahem, > >> >>>>> good question Misty (in the literal, not flattering, sense). > >> >>>>>> > >> >>>>>> I think hbase-connectors will be like hbase-thirdparty. The > latter > >> >>>>> includes netty, pb, guava and a few other bits and pieces so yeah, > >> >>>>> sometimes a netty upgrade or an improvement on our patch to pb > will > >> >>>> require > >> >>>>> us releasing all though we are fixing one lib only. Usually, if > >> >> bothering > >> >>>>> to make a release, we'll check for fixes or updates we can do in > the > >> >>>> other > >> >>>>> bundled components. > >> >>>>>> > >> >>>>>> On the rate of releases, I foresee a flurry of activity around > >> launch > >> >>>> as > >> >>>>> we fill missing bits and address critical bug fixes, but that then > >> it > >> >>>> will > >> >>>>> settle down to be boring, with just the occasional update. Thrift > >> and > >> >>>> REST > >> >>>>> have been stable for a good while now (not saying this is a good > >> >> thing). > >> >>>>> Our Sean just suggested moving mapreduce to connectors too -- an > >> >>>>> interesting idea -- and this has also been stable too (at least > >> until > >> >>>>> recently with the shading work). We should talk about the Spark > >> >> connector > >> >>>>> when it comes time. It might not be as stable as the others. > >> >>>>>> > >> >>>>>> On the compatibility guarantees, we'll semver it so if an > >> incompatible > >> >>>>> change in a connector or if the connectors have to change to > match a > >> >> new > >> >>>>> version of hbase, we'll make sure the hbase-connector version > >> number is > >> >>>>> changed appropriately. On the backend, what Mike says; connectors > >> use > >> >>>> HBase > >> >>>>> Public APIs (else they can't be moved to the hbase-connector > repo). > >> >>>>>> > >> >>>>>> S > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>>> > >> >>>>>>> On Tue, Jul 24, 2018, 9:41 PM Stack <st...@duboce.net> wrote: > >> >>>>>>> > >> >>>>>>>> Grand. I filed > https://issues.apache.org/jira/browse/HBASE-20934 > >> . > >> >>>>> Let me > >> >>>>>>>> have a go at making the easy one work first (the kafka proxy). > >> Lets > >> >>>>> see how > >> >>>>>>>> it goes. I'll report back here. > >> >>>>>>>> S > >> >>>>>>>> > >> >>>>>>>> On Tue, Jul 24, 2018 at 2:43 PM Sean Busbey <bus...@apache.org > > > >> >>>>> wrote: > >> >>>>>>>> > >> >>>>>>>>> Key functionality for the project's adoption should be in the > >> >>>>> project. > >> >>>>>>>>> Please do not suggest we donate things to Bahir. > >> >>>>>>>>> > >> >>>>>>>>> I apologize if this is brisk. I have had previous negative > >> >>>>> experiences > >> >>>>>>>>> with folks that span our communities trying to move work I > >> spent a > >> >>>>> lot > >> >>>>>>>>> of time contributing to within HBase over to Bahir in an > attempt > >> >>>> to > >> >>>>>>>>> bypass an agreed upon standard of quality. > >> >>>>>>>>> > >> >>>>>>>>> On Tue, Jul 24, 2018 at 3:38 PM, Artem Ervits < > >> >>>>> artemerv...@gmail.com> > >> >>>>>>>>> wrote: > >> >>>>>>>>>> Why not just donating the connector to > >> http://bahir.apache.org/ > >> >>>> ? > >> >>>>>>>>>> > >> >>>>>>>>>> On Tue, Jul 24, 2018, 12:51 PM Lars Francke < > >> >>>>> lars.fran...@gmail.com> > >> >>>>>>>>> wrote: > >> >>>>>>>>>> > >> >>>>>>>>>>> I'd love to have the Kafka Connector included. > >> >>>>>>>>>>> > >> >>>>>>>>>>> @Mike thanks so much for the contribution (and your planned > >> >>>> ones) > >> >>>>>>>>>>> > >> >>>>>>>>>>> I'm +1 on adding it to the core but I'm also +1 on having a > >> >>>>> separate > >> >>>>>>>>>>> repository under Apache governance > >> >>>>>>>>>>> > >> >>>>>>>>>>> On Tue, Jul 24, 2018 at 6:01 PM, Josh Elser < > >> els...@apache.org > >> >>>>> > >> >>>>>>>> wrote: > >> >>>>>>>>>>> > >> >>>>>>>>>>>> +1 to the great point by Duo about use of non-IA.Public > >> >>>> classes > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> +1 for Apache for the governance (although, I wouldn't care > >> >>>> if > >> >>>>> we > >> >>>>>>>> use > >> >>>>>>>>>>>> Github PRs to try to encourage more folks to contribute), a > >> >>>>> repo > >> >>>>>>>> with > >> >>>>>>>>> the > >> >>>>>>>>>>>> theme of "connectors" (to include Thrift, REST, and the > >> >>>> like). > >> >>>>> Spark > >> >>>>>>>>> too > >> >>>>>>>>>>> -- > >> >>>>>>>>>>>> I think we had suggested that prior, but it could be a > mental > >> >>>>>>>>> invention > >> >>>>>>>>>>> of > >> >>>>>>>>>>>> mine.. > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> On 7/24/18 10:16 AM, Hbase Janitor wrote: > >> >>>>>>>>>>>> > >> >>>>>>>>>>>>> Hi everyone, > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> I'm the author of the patch. A separate repo for all the > >> >>>>>>>> connectors > >> >>>>>>>>> is > >> >>>>>>>>>>> a > >> >>>>>>>>>>>>> great idea! I can make whatever changes necessary to the > >> >>>>> patch to > >> >>>>>>>>> help. > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> I have several other integration type projects like this > >> >>>>> planned. > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> Mike > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> On Tue, Jul 24, 2018, 00:03 Mike Drob <md...@apache.org> > >> >>>>> wrote: > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> I would be ok with all of the connectors in a single repo. > >> >>>>> Doing a > >> >>>>>>>>> repo > >> >>>>>>>>>>>>>> per > >> >>>>>>>>>>>>>> connector seems like a large amount of overhead work. > >> >>>>>>>>>>>>>> > >> >>>>>>>>>>>>>> On Mon, Jul 23, 2018, 9:12 PM Clay B. <c...@clayb.net> > >> >>>> wrote: > >> >>>>>>>>>>>>>> > >> >>>>>>>>>>>>>> [Non-binding] > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> I am all for the Kafka Connect(er) as indeed it makes > >> >>>> HBase > >> >>>>> "more > >> >>>>>>>>>>>>>>> relevant" and generates buzz to help me sell HBase > >> >>>> adoption > >> >>>>> in my > >> >>>>>>>>>>>>>>> endeavors. > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> Also, I would like to see a connectors repo a lot as I > >> >>>> would > >> >>>>>>>>> expect it > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>> can > >> >>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> make the HBase source and releases more obvious in what > is > >> >>>>>>>>> changing. > >> >>>>>>>>>>> Not > >> >>>>>>>>>>>>>>> to distract from Kafka, but Spark has in the past been a > >> >>>>> hang-up > >> >>>>>>>>> and > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>> seems > >> >>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> a good fit in such a repo too; as such, I would prefer > >> >>>>> Apache > >> >>>>>>>> over > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>> GitHub. > >> >>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> -Clay > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> On Mon, 23 Jul 2018, Andrew Purtell wrote: > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> Would we make a new repo called hbase-connectors and > move > >> >>>>> REST, > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> thrift, > >> >>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> and this new patch there? > >> >>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> I like this idea. We are already releasing > >> >>>>> hbase-thirdparty like > >> >>>>>>>>>>> this. > >> >>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> On Mon, Jul 23, 2018 at 5:47 PM Stack < > st...@duboce.net> > >> >>>>> wrote: > >> >>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> (Thanks for the good discussion) > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> Where we think 'outside of HBase' would be? > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> Github seems too 'remote' from project and from > Apache? > >> >>>>> Would > >> >>>>>>>> we > >> >>>>>>>>>>> make > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> a > >> >>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> new > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> repo called hbase-connectors and move REST, thrift, and > >> >>>>> this new > >> >>>>>>>>>>> patch > >> >>>>>>>>>>>>>>>>> there? > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> Thanks, > >> >>>>>>>>>>>>>>>>> S > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> On Mon, Jul 23, 2018 at 3:50 PM Josh Elser < > >> >>>>> els...@apache.org> > >> >>>>>>>>>>> wrote: > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> I'm -0 for including this into the main hbase tree. I > >> >>>>> feel like > >> >>>>>>>>>>> we've > >> >>>>>>>>>>>>>>>>>> made a bit of progress in cleaning up our core, and > >> >>>> this > >> >>>>>>>>> strikes me > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> as > >> >>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> a > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> step in the wrong direction. > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> At the same time, the integration seems nice enough > >> >>>> (for > >> >>>>> the > >> >>>>>>>>> same > >> >>>>>>>>>>>>>>>>>> reasons Andrew points out). Is there a reason this > >> >>>>> couldn't > >> >>>>>>>>> exist > >> >>>>>>>>>>>>>>>>>> outside of HBase (at the ASF or otherwise)? Given a > >> >>>> quick > >> >>>>>>>>> glance at > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> the > >> >>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> patch, it would be quite trivial to keep separate (just > >> >>>>> requires > >> >>>>>>>>> some > >> >>>>>>>>>>>>>>>>>> heavier scripting to get it off the ground that the > >> >>>> HBase > >> >>>>>>>>> scripts > >> >>>>>>>>>>> do > >> >>>>>>>>>>>>>>>>>> setup for). I feel like that will decrease our debt > >> >>>>> while we > >> >>>>>>>>> see if > >> >>>>>>>>>>>>>>>>>> people start using it. Our API should be more than > >> >>>> stable > >> >>>>>>>>> enough to > >> >>>>>>>>>>>>>>>>>> prevent any worry about drift happening from core to > >> >>>> this > >> >>>>>>>>> project. > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> On 7/23/18 6:35 PM, Stack wrote: > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>>> We have a very nice contrib sitting up in > HBASE-15320 > >> >>>>> which > >> >>>>>>>>> via a > >> >>>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> proxy > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> -- > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>>> so minimal dependencies -- adds source and sink for > >> >>>>> Kafka > >> >>>>>>>>> Connect. > >> >>>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> It > >> >>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> is > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> nicely contained inside two new hbase-kafka-* > modules. > >> >>>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>>> We good w/ taking on this new feature? > >> >>>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>>> It looks good to me. Check it out up on > HBASE-15320. I > >> >>>>> was > >> >>>>>>>>> going > >> >>>>>>>>>>> to > >> >>>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> commit > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>>> to tip of branch-2 so it'd show up in hbase-2.2.x > >> >>>>> unless you > >> >>>>>>>>> all > >> >>>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> want > >> >>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> some > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>>> backporting action going on. > >> >>>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>>> S > >> >>>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> -- > >> >>>>>>>>>>>>>>>> Best regards, > >> >>>>>>>>>>>>>>>> Andrew > >> >>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> Words like orphans lost among the crosstalk, meaning > torn > >> >>>>> from > >> >>>>>>>>>>> truth's > >> >>>>>>>>>>>>>>>> decrepit hands > >> >>>>>>>>>>>>>>>> - A23, Crosstalk > >> >>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>> > >> >>>>>>>>>>>>>> > >> >>>>>>>>>>>>> > >> >>>>>>>>>>> > >> >>>>>>>>> > >> >>>>>>>> > >> >>>>> > >> >>>> > >> >>> > >> >> > >> > > >> > > > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
