We should be able to request just one alternate repo from INFRA, and put a top
hierarchical level in it that doesn’t include a maven pom. As far as maven and
clients are concerned, it
just increases by 1 the path length to the root of the repo.
On 3/31/17, 10:30 AM, "zeo...@gmail.com" <zeo...@gmail.com> wrote:
Once we agree on a repo location to host this, I would be happy to put
together the package and update our environments to use bro-pkg to install
the plugin. I have created METRON-813
<https://issues.apache.org/jira/browse/METRON-813> to track this and
changed METRON-348 <https://issues.apache.org/jira/browse/METRON-348> to be
a sub-task.
Otto - the bro packages model doesn't allow colocation with anything else.
That said, if we have two similar situations, and given the INFRA example
<https://issues.apache.org/jira/browse/INFRA-7060> Casey linked to before
was requesting 9 repos, perhaps we just request two repos. Would someone
else mind putting that request in?
Jon
On Fri, Mar 31, 2017 at 12:49 PM Otto Fowler <ottobackwa...@gmail.com>
wrote:
Could we create a separate repo for more than on thing? like put … um
let’s say
a maven plugin and the bro plugin?
On March 31, 2017 at 12:30:25, Nick Allen (n...@nickallen.org) wrote:
I agree with everything that I've read.
One of the guys from Bro had contacted me a while back, letting me know
that the packaging mechanism in Bro was ready for public consumption. I
just have not had cycles to do anything with it yet. They are not wanting
to host any of the plugins.
I thought the package mechanism requires that a package live within its own
repo (which Casey confirmed). This put me in a bind on how to tackle
this. I don't want to personally host the plugin in my own Github repo. I
would prefer that we host it in a community repo; either Bro or Metron.
Since Bro is moving away from hosting their own plugins, that leaves
Metron.
It would be great if we could create a separate repo for the plugin. That
solves the challenge of using the packaging mechanism.
We do need to reconcile what is in bro/bro-plugins and what is in Metron.
There are some enhancements that I and others have made that never made it
back into Metron. They never made it back, because the original plan was
just to switch to using the plugin from bro/bro-plugins before the idea of
a packaging mechanism hit Bro. Reconciling should be fairly easy to see by
just doing a diff.
It would be great if others want to take on any of that work. I would be
glad to offer any support that you need. Thanks, Jon!
On Thu, Mar 30, 2017 at 11:20 PM, zeo...@gmail.com <zeo...@gmail.com>
wrote:
> Ok, great.
>
> I agree, I definitely want to hear from Nick on the topic. My team is
> currently looking into enhancing the plugin as well to potentially allow
> sending to multiple clusters, investigating some issues we see when our
bro
> cluster is under load, turn it into a package, etc.
>
> The work you just did was on our to do list as well so I'm very excited
to
> see it come through.
>
> Jon
>
> On Thu, Mar 30, 2017, 11:16 PM Casey Stella <ceste...@gmail.com> wrote:
>
> I *think* it's possible. People do ask for mirrors of directories from
> time to time (see https://issues.apache.org/jira/browse/INFRA-7060). If
> we
> think this is a good idea, we can pose it to INFRA as a request. I'd love
> to see us be able to use the bro packaging infrastructure and get more
> visibility for the plugin.
>
> I'd be particularly interested in Nick's opinion on this, though.
>
> On Thu, Mar 30, 2017 at 11:12 PM, zeo...@gmail.com <zeo...@gmail.com>
> wrote:
>
> > You can version packages -
> > http://bro-package-manager.readthedocs.io/en/stable/
> package.html#package-
> > versioning
> >
> > I agree that having a separate repo provided by Apache would be
optimal,
> I
> > just don't know the process for that or if it was even reasonable to
> > suggest.
> >
> > Jon
> >
> > On Thu, Mar 30, 2017, 11:01 PM Casey Stella <ceste...@gmail.com> wrote:
> >
> > > Looking at the bro packages, it appears that bro is expecting things
to
> > be
> > > its own git repository. I wonder if we could either request INFRA
> > provide
> > > another repo for the bro-kafka plugin and integrate it into metron as
a
> > git
> > > submodule *or* if we could request INFRA to create a github mirror of
> the
> > > metron-sensors/bro-kafka-plugin directory. I'm not sure how viable
> > either
> > > of those options are, frankly.
> > >
> > > One thing that I didn't see is how do you specify a particular
release
> of
> > > the plugin that you want to install? For us, we'd want to release the
> > > plugin along with the product. I didn't quite see how you'd push
> > releases
> > > for bro plugins.
> > >
> > > On Thu, Mar 30, 2017 at 10:49 PM, Casey Stella <ceste...@gmail.com>
> > wrote:
> > >
> > > > So, I do agree with the concern. Is there a way to host the package
> > > > within Metron? I definitely would like to see the modifications at
> > > > https://github.com/bro/bro-plugins/commit/b9f1f35415cb0db
> > > > 065348da0a5043a8353b4a0a8 brought back into Metron and I'd love for
> us
> > to
> > > > host the plugin.
> > > >
> > > > Thoughts?
> > > >
> > > >
> > > > On Thu, Mar 30, 2017 at 9:09 PM, zeo...@gmail.com <zeo...@gmail.com>
> > > > wrote:
> > > >
> > > >> Today I was taking a look at METRON-812
> > > >> <https://issues.apache.org/jira/browse/METRON-812>, which made me
> > > recall
> > > >> some conversations from a while back regarding where the bro kafka
> > > plugin
> > > >> should ultimately live, and how to update it.
> > > >>
> > > >> Back in METRON-348 <https://issues.apache.org/
> jira/browse/METRON-348>
> > I
> > > >> brought up the fact that some important changes
> > > >> <https://github.com/bro/bro-plugins/commit/b9f1f35415cb0db06
> > > >> 5348da0a5043a8353b4a0a8>
> > > >> were made to the externally hosted version of the kafka plugin,
and
> > were
> > > >> never introduced to Metron's hosted version (i.e. the one we use
> > > >> <https://github.com/apache/incubator-metron/blob/master/metr
> > > >> on-deployment/roles/bro/tasks/bro-plugin-kafka.yml>
> > > >> in vagrant when bro is installed). The conversation went down the
> > route
> > > >> of
> > > >> discussing whether or not the bro kafka plugin code should
continue
> to
> > > >> live
> > > >> in Metron in the first place. Now, with METRON-812, I see us
> further
> > > >> muddying the waters of where to go for the right plugin, as our
> > version
> > > is
> > > >> still missing the public changes but adds some very important new
> > > >> functionality.
> > > >>
> > > >> I'd like to bring up the idea of using bro's packages
> > > >> <https://github.com/bro/packages> framework, released in late 2016
> > > >> <http://blog.bro.org/2016/10/introducing-bro-package-manager.html>
> > > >> (additional
> > > >> documentation here <
> > > http://bro-package-manager.readthedocs.io/en/stable/
> > > >> >),
> > > >> as a potential place for this to be hosted/referenced. This is a
> > simple
> > > >> and supported method (funded by Mozilla
> > > >> <https://blog.mozilla.org/blog/2015/12/10/mozilla-open-sourc
> > > >> e-support-first-awards-made/>)
> > > >> to install and uninstall bro scripts, plugins, etc., and it also
> > allows
> > > us
> > > >> to continue to have enough control over updates to the plugin so
> that
> > it
> > > >> will not slow down Metron development by having it as a dependency
> > > >> (resolving both of Casey's concerns noted here
> > > >> <https://issues.apache.org/jira/browse/METRON-348?focusedCom
> > > >> mentId=15391865&page=com.atlassian.jira.plugin.system.
> > > >> issuetabpanels:comment-tabpanel#comment-15391865>,
> > > >> and I think this solution is supported by Nick's comments here
> > > >> <https://issues.apache.org/jira/browse/METRON-348?focusedCom
> > > >> mentId=15391872&page=com.atlassian.jira.plugin.system.
> > > >> issuetabpanels:comment-tabpanel#comment-15391872>
> > > >> as
> > > >> well).
> > > >>
> > > >> The only thing I'm not sure about is where to host the plugin
itself
> -
> > > my
> > > >> first thought would be Nick's github <https://github.com/nickwallen
> >,
> > > as
> > > >> he
> > > >> really kicked off this effort, but maybe we can think of something
> > > better.
> > > >>
> > > >> Is this approach of interest to anybody? It is extremely simple to
> > put
> > > >> together - I was able to throw one together
> > > >> <https://github.com/bro/packages/blob/master/
> jonzeolla/bro-pkg.index>
> > > and
> > > >> get it working with a fresh bro 2.5 install when attending the
> brocon
> > > talk
> > > >> <https://www.bro.org/brocon2016/brocon2016_abstracts.html#
> > > >> bro-packagemanager>
> > > >> (recording <https://www.youtube.com/watch?v=9RFfPJeGkcE>, slides
> > > >> <https://www.bro.org/brocon2016/slides/hall_bpm.pdf>) that
> introduced
> > > >> this
> > > >> to me in the first place.
> > > >>
> > > >> Jon
> > > >> --
> > > >>
> > > >> Jon
> > > >>
> > > >
> > > >
> > >
> > --
> >
> > Jon
> >
>
> --
>
> Jon
>
--
Jon
