IANAL, but to my knowledge, the bro-kafka plugin hosted inside of the bro project and the bro-kafka plugin hosted inside of Metron are two separate plugins. Indeed, these two plugins have diverged in capabilities. Nick contributed them both to two separate OSS projects, which is totally fine. If you want to submit a PR to correct the multithreading bug that exists in the Metron bro-kafka plugin, then I think that's perfectly fine. I wouldn't think of it as making them equal, but rather correcting a bug in the Metron bro-kafka plugin.
It might be good to pose the question on dev with the subject prefaced with [MENTORS]. They will likely have different opinions. One bit of detail not mentioned yet is that bro-kafka plugin inside of bro is 3-clause BSD licensed. The question in my mind is if we submit this correction whether it falls under bundling (which would necessitate a line in our LICENSE file), is a derivative work (which is kosher because it's BSD licensed) or whether it's just the contribution of a piece of functionality that another piece of software has. Casey On Wed, Apr 5, 2017 at 1:29 PM, zeo...@gmail.com <zeo...@gmail.com> wrote: > Okay great, thanks. Would you mind throwing in an INFRA ticket for the new > repo? I can take it all from there. > > Does anybody know if we have ASF resources to help answer the above legal > question? > > Jon > > On Wed, Apr 5, 2017 at 1:26 PM Nick Allen <n...@nickallen.org> wrote: > > > (1) I am not sure if licensing is a problem here. > > > > (2) I am OK with whatever we need to get this effort done and under ASF. > > > > > > On Wed, Apr 5, 2017 at 1:12 PM, zeo...@gmail.com <zeo...@gmail.com> > wrote: > > > > > I'm working on this > > > <https://github.com/JonZeolla/incubator-metron/tree/METRON-348> in > > > preparation for the new repo and migration to a package. It looks like > > in > > > bro-plugins COPYING > > > <https://github.com/bro/bro-plugins/blob/master/kafka/COPYING> > > attributes > > > to Nick, but in our version COPYING > > > <https://github.com/apache/incubator-metron/tree/master/ > > > metron-sensors/bro-plugin-kafka/COPYING> > > > points to the Apache License. Same with MAINTAINER (this > > > <https://github.com/apache/incubator-metron/blob/master/ > > > metron-sensors/bro-plugin-kafka/MAINTAINER> > > > vs this <https://github.com/bro/bro-plugins/blob/master/kafka/ > MAINTAINER > > > >). > > > I assume when we package this up and host it in Apache we need to give > it > > > the Apache license, and point to Metron for MAINTAINER. My questions > > are: > > > > > > 1. Is there any legal/licensing concern here? I am taking changes > from > > > the bro-plugins version and pulling it into the Apache-hosted code. > > IANAL > > > 2. Nick - are you OK with these changes? > > > > > > Jon > > > > > > On Mon, Apr 3, 2017 at 3:50 PM zeo...@gmail.com <zeo...@gmail.com> > > wrote: > > > > > > > Can someone on the PMC submit a ticket to INFRA? It looks like > > > > <https://www.apache.org/dev/infra-contact> committers aren't > supposed > > > to. > > > > > > > > Jon > > > > > > > > On Fri, Mar 31, 2017 at 4:23 PM zeo...@gmail.com <zeo...@gmail.com> > > > wrote: > > > > > > > > I would be happy to try it again but I attempted to do that before > with > > > > bro packages and it failed to be able to handle it. I also tried > using > > > > branches of a repo with bro but that similarly failed (and was a > pretty > > > bad > > > > idea to start with). > > > > > > > > Jon > > > > > > > > On Fri, Mar 31, 2017, 3:24 PM Matt Foley <ma...@apache.org> wrote: > > > > > > > > We should be able to request just one alternate repo from INFRA, and > > put > > > a > > > > top hierarchical level in it that doesn’t include a maven pom. As > far > > as > > > > maven and clients are concerned, it > > > > > > > > just increases by 1 the path length to the root of the repo. > > > > > > > > On 3/31/17, 10:30 AM, "zeo...@gmail.com" <zeo...@gmail.com> wrote: > > > > > > > > Once we agree on a repo location to host this, I would be happy > to > > > put > > > > together the package and update our environments to use bro-pkg > to > > > > install > > > > the plugin. I have created METRON-813 > > > > <https://issues.apache.org/jira/browse/METRON-813> to track this > > and > > > > changed METRON-348 < > > https://issues.apache.org/jira/browse/METRON-348 > > > > > > > > to be > > > > a sub-task. > > > > > > > > Otto - the bro packages model doesn't allow colocation with > > anything > > > > else. > > > > That said, if we have two similar situations, and given the INFRA > > > > example > > > > <https://issues.apache.org/jira/browse/INFRA-7060> Casey linked > to > > > > before > > > > was requesting 9 repos, perhaps we just request two repos. Would > > > > someone > > > > else mind putting that request in? > > > > > > > > Jon > > > > > > > > On Fri, Mar 31, 2017 at 12:49 PM Otto Fowler < > > > ottobackwa...@gmail.com> > > > > wrote: > > > > > > > > Could we create a separate repo for more than on thing? like > put … > > > um > > > > let’s say > > > > a maven plugin and the bro plugin? > > > > > > > > > > > > > > > > On March 31, 2017 at 12:30:25, Nick Allen (n...@nickallen.org) > > > wrote: > > > > > > > > I agree with everything that I've read. > > > > > > > > One of the guys from Bro had contacted me a while back, letting > me > > > know > > > > that the packaging mechanism in Bro was ready for public > > > consumption. I > > > > just have not had cycles to do anything with it yet. They are not > > > > wanting > > > > to host any of the plugins. > > > > > > > > I thought the package mechanism requires that a package live > within > > > > its own > > > > repo (which Casey confirmed). This put me in a bind on how to > > tackle > > > > this. I don't want to personally host the plugin in my own Github > > > > repo. I > > > > would prefer that we host it in a community repo; either Bro or > > > Metron. > > > > Since Bro is moving away from hosting their own plugins, that > > leaves > > > > Metron. > > > > > > > > It would be great if we could create a separate repo for the > > plugin. > > > > That > > > > solves the challenge of using the packaging mechanism. > > > > > > > > We do need to reconcile what is in bro/bro-plugins and what is in > > > > Metron. > > > > There are some enhancements that I and others have made that > never > > > > made it > > > > back into Metron. They never made it back, because the original > > plan > > > > was > > > > just to switch to using the plugin from bro/bro-plugins before > the > > > > idea of > > > > a packaging mechanism hit Bro. Reconciling should be fairly easy > to > > > > see by > > > > just doing a diff. > > > > > > > > It would be great if others want to take on any of that work. I > > would > > > > be > > > > glad to offer any support that you need. Thanks, Jon! > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Mar 30, 2017 at 11:20 PM, zeo...@gmail.com < > > zeo...@gmail.com > > > > > > > > wrote: > > > > > > > > > Ok, great. > > > > > > > > > > I agree, I definitely want to hear from Nick on the topic. My > > team > > > is > > > > > currently looking into enhancing the plugin as well to > > potentially > > > > allow > > > > > sending to multiple clusters, investigating some issues we see > > when > > > > our > > > > bro > > > > > cluster is under load, turn it into a package, etc. > > > > > > > > > > The work you just did was on our to do list as well so I'm very > > > > excited > > > > to > > > > > see it come through. > > > > > > > > > > Jon > > > > > > > > > > On Thu, Mar 30, 2017, 11:16 PM Casey Stella < > ceste...@gmail.com> > > > > wrote: > > > > > > > > > > I *think* it's possible. People do ask for mirrors of > directories > > > > from > > > > > time to time (see > > https://issues.apache.org/jira/browse/INFRA-7060 > > > ). > > > > If > > > > > we > > > > > think this is a good idea, we can pose it to INFRA as a > request. > > > I'd > > > > love > > > > > to see us be able to use the bro packaging infrastructure and > get > > > > more > > > > > visibility for the plugin. > > > > > > > > > > I'd be particularly interested in Nick's opinion on this, > though. > > > > > > > > > > On Thu, Mar 30, 2017 at 11:12 PM, zeo...@gmail.com < > > > zeo...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > You can version packages - > > > > > > http://bro-package-manager.readthedocs.io/en/stable/ > > > > > package.html#package- > > > > > > versioning > > > > > > > > > > > > I agree that having a separate repo provided by Apache would > be > > > > optimal, > > > > > I > > > > > > just don't know the process for that or if it was even > > reasonable > > > > to > > > > > > suggest. > > > > > > > > > > > > Jon > > > > > > > > > > > > On Thu, Mar 30, 2017, 11:01 PM Casey Stella < > > ceste...@gmail.com> > > > > wrote: > > > > > > > > > > > > > Looking at the bro packages, it appears that bro is > expecting > > > > things > > > > to > > > > > > be > > > > > > > its own git repository. I wonder if we could either request > > > INFRA > > > > > > provide > > > > > > > another repo for the bro-kafka plugin and integrate it into > > > > metron as > > > > a > > > > > > git > > > > > > > submodule *or* if we could request INFRA to create a github > > > > mirror of > > > > > the > > > > > > > metron-sensors/bro-kafka-plugin directory. I'm not sure > how > > > > viable > > > > > > either > > > > > > > of those options are, frankly. > > > > > > > > > > > > > > One thing that I didn't see is how do you specify a > > particular > > > > release > > > > > of > > > > > > > the plugin that you want to install? For us, we'd want to > > > > release the > > > > > > > plugin along with the product. I didn't quite see how you'd > > > push > > > > > > releases > > > > > > > for bro plugins. > > > > > > > > > > > > > > On Thu, Mar 30, 2017 at 10:49 PM, Casey Stella < > > > > ceste...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > So, I do agree with the concern. Is there a way to host > the > > > > package > > > > > > > > within Metron? I definitely would like to see the > > > > modifications at > > > > > > > > https://github.com/bro/bro- > plugins/commit/b9f1f35415cb0db > > > > > > > > 065348da0a5043a8353b4a0a8 brought back into Metron and > I'd > > > > love for > > > > > us > > > > > > to > > > > > > > > host the plugin. > > > > > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Mar 30, 2017 at 9:09 PM, zeo...@gmail.com < > > > > zeo...@gmail.com> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > >> Today I was taking a look at METRON-812 > > > > > > > >> <https://issues.apache.org/jira/browse/METRON-812>, > which > > > > made me > > > > > > > recall > > > > > > > >> some conversations from a while back regarding where the > > bro > > > > kafka > > > > > > > plugin > > > > > > > >> should ultimately live, and how to update it. > > > > > > > >> > > > > > > > >> Back in METRON-348 <https://issues.apache.org/ > > > > > jira/browse/METRON-348> > > > > > > I > > > > > > > >> brought up the fact that some important changes > > > > > > > >> <https://github.com/bro/bro-plugins/commit/ > > > b9f1f35415cb0db06 > > > > > > > >> 5348da0a5043a8353b4a0a8> > > > > > > > >> were made to the externally hosted version of the kafka > > > > plugin, > > > > and > > > > > > were > > > > > > > >> never introduced to Metron's hosted version (i.e. the > one > > we > > > > use > > > > > > > >> <https://github.com/apache/ > incubator-metron/blob/master/ > > > metr > > > > > > > >> on-deployment/roles/bro/tasks/bro-plugin-kafka.yml> > > > > > > > >> in vagrant when bro is installed). The conversation went > > > down > > > > the > > > > > > route > > > > > > > >> of > > > > > > > >> discussing whether or not the bro kafka plugin code > should > > > > continue > > > > > to > > > > > > > >> live > > > > > > > >> in Metron in the first place. Now, with METRON-812, I > see > > us > > > > > further > > > > > > > >> muddying the waters of where to go for the right plugin, > > as > > > > our > > > > > > version > > > > > > > is > > > > > > > >> still missing the public changes but adds some very > > > important > > > > new > > > > > > > >> functionality. > > > > > > > >> > > > > > > > >> I'd like to bring up the idea of using bro's packages > > > > > > > >> <https://github.com/bro/packages> framework, released > in > > > > late 2016 > > > > > > > >> < > > > > http://blog.bro.org/2016/10/introducing-bro-package-manager.html> > > > > > > > >> (additional > > > > > > > >> documentation here < > > > > > > > http://bro-package-manager.readthedocs.io/en/stable/ > > > > > > > >> >), > > > > > > > >> as a potential place for this to be hosted/referenced. > > This > > > > is a > > > > > > simple > > > > > > > >> and supported method (funded by Mozilla > > > > > > > >> <https://blog.mozilla.org/blog/2015/12/10/mozilla-open- > > > sourc > > > > > > > >> e-support-first-awards-made/>) > > > > > > > >> to install and uninstall bro scripts, plugins, etc., and > > it > > > > also > > > > > > allows > > > > > > > us > > > > > > > >> to continue to have enough control over updates to the > > > plugin > > > > so > > > > > that > > > > > > it > > > > > > > >> will not slow down Metron development by having it as a > > > > dependency > > > > > > > >> (resolving both of Casey's concerns noted here > > > > > > > >> <https://issues.apache.org/jira/browse/METRON-348? > > > focusedCom > > > > > > > >> mentId=15391865&page=com.atlassian.jira.plugin.system. > > > > > > > >> issuetabpanels:comment-tabpanel#comment-15391865>, > > > > > > > >> and I think this solution is supported by Nick's > comments > > > here > > > > > > > >> <https://issues.apache.org/jira/browse/METRON-348? > > > focusedCom > > > > > > > >> mentId=15391872&page=com.atlassian.jira.plugin.system. > > > > > > > >> issuetabpanels:comment-tabpanel#comment-15391872> > > > > > > > >> as > > > > > > > >> well). > > > > > > > >> > > > > > > > >> The only thing I'm not sure about is where to host the > > > plugin > > > > itself > > > > > - > > > > > > > my > > > > > > > >> first thought would be Nick's github < > > > > https://github.com/nickwallen > > > > > >, > > > > > > > as > > > > > > > >> he > > > > > > > >> really kicked off this effort, but maybe we can think of > > > > something > > > > > > > better. > > > > > > > >> > > > > > > > >> Is this approach of interest to anybody? It is extremely > > > > simple to > > > > > > put > > > > > > > >> together - I was able to throw one together > > > > > > > >> <https://github.com/bro/packages/blob/master/ > > > > > jonzeolla/bro-pkg.index> > > > > > > > and > > > > > > > >> get it working with a fresh bro 2.5 install when > attending > > > the > > > > > brocon > > > > > > > talk > > > > > > > >> < > > https://www.bro.org/brocon2016/brocon2016_abstracts.html# > > > > > > > >> bro-packagemanager> > > > > > > > >> (recording <https://www.youtube.com/watch?v=9RFfPJeGkcE > >, > > > > slides > > > > > > > >> <https://www.bro.org/brocon2016/slides/hall_bpm.pdf>) > > that > > > > > introduced > > > > > > > >> this > > > > > > > >> to me in the first place. > > > > > > > >> > > > > > > > >> Jon > > > > > > > >> -- > > > > > > > >> > > > > > > > >> Jon > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Jon > > > > > > > > > > > > > > > > -- > > > > > > > > > > Jon > > > > > > > > > > > > > -- > > > > > > > > Jon > > > > > > > > > > > > > > > > -- > > > > > > > > Jon > > > > > > > > -- > > > > > > > > Jon > > > > > > > -- > > > > > > Jon > > > > > > -- > > Jon >
