Browsing https://git.apache.org/ shows lots of examples. A few are quite
prolific.
On 4/5/17, 1:00 PM, "Nick Allen" <n...@nickallen.org> wrote:
Does anyone know any other Apache projects that are using multiple repos?
I'd like to see what they've done just so we don't break convention.
On Wed, Apr 5, 2017 at 3:22 PM, Nick Allen <n...@nickallen.org> wrote:
> Yes, I will open an INFRA ticket. Just give me a little time to research
> what we need.
>
> On Wed, Apr 5, 2017 at 1:29 PM, zeo...@gmail.com <zeo...@gmail.com> wrote:
>
>> Okay great, thanks. Would you mind throwing in an INFRA ticket for the
>> new
>> repo? I can take it all from there.
>>
>> Does anybody know if we have ASF resources to help answer the above legal
>> question?
>>
>> Jon
>>
>> On Wed, Apr 5, 2017 at 1:26 PM Nick Allen <n...@nickallen.org> wrote:
>>
>> > (1) I am not sure if licensing is a problem here.
>> >
>> > (2) I am OK with whatever we need to get this effort done and under
ASF.
>> >
>> >
>> > On Wed, Apr 5, 2017 at 1:12 PM, zeo...@gmail.com <zeo...@gmail.com>
>> wrote:
>> >
>> > > I'm working on this
>> > > <https://github.com/JonZeolla/incubator-metron/tree/METRON-348> in
>> > > preparation for the new repo and migration to a package. It looks
>> like
>> > in
>> > > bro-plugins COPYING
>> > > <https://github.com/bro/bro-plugins/blob/master/kafka/COPYING>
>> > attributes
>> > > to Nick, but in our version COPYING
>> > > <https://github.com/apache/incubator-metron/tree/master/
>> > > metron-sensors/bro-plugin-kafka/COPYING>
>> > > points to the Apache License. Same with MAINTAINER (this
>> > > <https://github.com/apache/incubator-metron/blob/master/
>> > > metron-sensors/bro-plugin-kafka/MAINTAINER>
>> > > vs this <https://github.com/bro/bro-plugins/blob/master/kafka/MAINTA
>> INER
>> > > >).
>> > > I assume when we package this up and host it in Apache we need to
>> give it
>> > > the Apache license, and point to Metron for MAINTAINER. My questions
>> > are:
>> > >
>> > > 1. Is there any legal/licensing concern here? I am taking changes
>> from
>> > > the bro-plugins version and pulling it into the Apache-hosted code.
>> > IANAL
>> > > 2. Nick - are you OK with these changes?
>> > >
>> > > Jon
>> > >
>> > > On Mon, Apr 3, 2017 at 3:50 PM zeo...@gmail.com <zeo...@gmail.com>
>> > wrote:
>> > >
>> > > > Can someone on the PMC submit a ticket to INFRA? It looks like
>> > > > <https://www.apache.org/dev/infra-contact> committers aren't
>> supposed
>> > > to.
>> > > >
>> > > > Jon
>> > > >
>> > > > On Fri, Mar 31, 2017 at 4:23 PM zeo...@gmail.com <zeo...@gmail.com>
>> > > wrote:
>> > > >
>> > > > I would be happy to try it again but I attempted to do that before
>> with
>> > > > bro packages and it failed to be able to handle it. I also tried
>> using
>> > > > branches of a repo with bro but that similarly failed (and was a
>> pretty
>> > > bad
>> > > > idea to start with).
>> > > >
>> > > > Jon
>> > > >
>> > > > On Fri, Mar 31, 2017, 3:24 PM Matt Foley <ma...@apache.org> wrote:
>> > > >
>> > > > We should be able to request just one alternate repo from INFRA,
and
>> > put
>> > > a
>> > > > top hierarchical level in it that doesn’t include a maven pom. As
>> far
>> > as
>> > > > maven and clients are concerned, it
>> > > >
>> > > > just increases by 1 the path length to the root of the repo.
>> > > >
>> > > > On 3/31/17, 10:30 AM, "zeo...@gmail.com" <zeo...@gmail.com> wrote:
>> > > >
>> > > > Once we agree on a repo location to host this, I would be happy
>> to
>> > > put
>> > > > together the package and update our environments to use bro-pkg
>> to
>> > > > install
>> > > > the plugin. I have created METRON-813
>> > > > <https://issues.apache.org/jira/browse/METRON-813> to track
>> this
>> > and
>> > > > changed METRON-348 <
>> > https://issues.apache.org/jira/browse/METRON-348
>> > > >
>> > > > to be
>> > > > a sub-task.
>> > > >
>> > > > Otto - the bro packages model doesn't allow colocation with
>> > anything
>> > > > else.
>> > > > That said, if we have two similar situations, and given the
>> INFRA
>> > > > example
>> > > > <https://issues.apache.org/jira/browse/INFRA-7060> Casey
>> linked to
>> > > > before
>> > > > was requesting 9 repos, perhaps we just request two repos.
>> Would
>> > > > someone
>> > > > else mind putting that request in?
>> > > >
>> > > > Jon
>> > > >
>> > > > On Fri, Mar 31, 2017 at 12:49 PM Otto Fowler <
>> > > ottobackwa...@gmail.com>
>> > > > wrote:
>> > > >
>> > > > Could we create a separate repo for more than on thing? like
>> put …
>> > > um
>> > > > let’s say
>> > > > a maven plugin and the bro plugin?
>> > > >
>> > > >
>> > > >
>> > > > On March 31, 2017 at 12:30:25, Nick Allen (n...@nickallen.org)
>> > > wrote:
>> > > >
>> > > > I agree with everything that I've read.
>> > > >
>> > > > One of the guys from Bro had contacted me a while back, letting
>> me
>> > > know
>> > > > that the packaging mechanism in Bro was ready for public
>> > > consumption. I
>> > > > just have not had cycles to do anything with it yet. They are
>> not
>> > > > wanting
>> > > > to host any of the plugins.
>> > > >
>> > > > I thought the package mechanism requires that a package live
>> within
>> > > > its own
>> > > > repo (which Casey confirmed). This put me in a bind on how to
>> > tackle
>> > > > this. I don't want to personally host the plugin in my own
>> Github
>> > > > repo. I
>> > > > would prefer that we host it in a community repo; either Bro or
>> > > Metron.
>> > > > Since Bro is moving away from hosting their own plugins, that
>> > leaves
>> > > > Metron.
>> > > >
>> > > > It would be great if we could create a separate repo for the
>> > plugin.
>> > > > That
>> > > > solves the challenge of using the packaging mechanism.
>> > > >
>> > > > We do need to reconcile what is in bro/bro-plugins and what is
>> in
>> > > > Metron.
>> > > > There are some enhancements that I and others have made that
>> never
>> > > > made it
>> > > > back into Metron. They never made it back, because the original
>> > plan
>> > > > was
>> > > > just to switch to using the plugin from bro/bro-plugins before
>> the
>> > > > idea of
>> > > > a packaging mechanism hit Bro. Reconciling should be fairly
>> easy to
>> > > > see by
>> > > > just doing a diff.
>> > > >
>> > > > It would be great if others want to take on any of that work. I
>> > would
>> > > > be
>> > > > glad to offer any support that you need. Thanks, Jon!
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Thu, Mar 30, 2017 at 11:20 PM, zeo...@gmail.com <
>> > zeo...@gmail.com
>> > > >
>> > > > wrote:
>> > > >
>> > > > > Ok, great.
>> > > > >
>> > > > > I agree, I definitely want to hear from Nick on the topic. My
>> > team
>> > > is
>> > > > > currently looking into enhancing the plugin as well to
>> > potentially
>> > > > allow
>> > > > > sending to multiple clusters, investigating some issues we
see
>> > when
>> > > > our
>> > > > bro
>> > > > > cluster is under load, turn it into a package, etc.
>> > > > >
>> > > > > The work you just did was on our to do list as well so I'm
>> very
>> > > > excited
>> > > > to
>> > > > > see it come through.
>> > > > >
>> > > > > Jon
>> > > > >
>> > > > > On Thu, Mar 30, 2017, 11:16 PM Casey Stella <
>> ceste...@gmail.com>
>> > > > wrote:
>> > > > >
>> > > > > I *think* it's possible. People do ask for mirrors of
>> directories
>> > > > from
>> > > > > time to time (see
>> > https://issues.apache.org/jira/browse/INFRA-7060
>> > > ).
>> > > > If
>> > > > > we
>> > > > > think this is a good idea, we can pose it to INFRA as a
>> request.
>> > > I'd
>> > > > love
>> > > > > to see us be able to use the bro packaging infrastructure and
>> get
>> > > > more
>> > > > > visibility for the plugin.
>> > > > >
>> > > > > I'd be particularly interested in Nick's opinion on this,
>> though.
>> > > > >
>> > > > > On Thu, Mar 30, 2017 at 11:12 PM, zeo...@gmail.com <
>> > > zeo...@gmail.com
>> > > > >
>> > > > > wrote:
>> > > > >
>> > > > > > You can version packages -
>> > > > > > http://bro-package-manager.readthedocs.io/en/stable/
>> > > > > package.html#package-
>> > > > > > versioning
>> > > > > >
>> > > > > > I agree that having a separate repo provided by Apache
>> would be
>> > > > optimal,
>> > > > > I
>> > > > > > just don't know the process for that or if it was even
>> > reasonable
>> > > > to
>> > > > > > suggest.
>> > > > > >
>> > > > > > Jon
>> > > > > >
>> > > > > > On Thu, Mar 30, 2017, 11:01 PM Casey Stella <
>> > ceste...@gmail.com>
>> > > > wrote:
>> > > > > >
>> > > > > > > Looking at the bro packages, it appears that bro is
>> expecting
>> > > > things
>> > > > to
>> > > > > > be
>> > > > > > > its own git repository. I wonder if we could either
>> request
>> > > INFRA
>> > > > > > provide
>> > > > > > > another repo for the bro-kafka plugin and integrate it
>> into
>> > > > metron as
>> > > > a
>> > > > > > git
>> > > > > > > submodule *or* if we could request INFRA to create a
>> github
>> > > > mirror of
>> > > > > the
>> > > > > > > metron-sensors/bro-kafka-plugin directory. I'm not sure
>> how
>> > > > viable
>> > > > > > either
>> > > > > > > of those options are, frankly.
>> > > > > > >
>> > > > > > > One thing that I didn't see is how do you specify a
>> > particular
>> > > > release
>> > > > > of
>> > > > > > > the plugin that you want to install? For us, we'd want to
>> > > > release the
>> > > > > > > plugin along with the product. I didn't quite see how
>> you'd
>> > > push
>> > > > > > releases
>> > > > > > > for bro plugins.
>> > > > > > >
>> > > > > > > On Thu, Mar 30, 2017 at 10:49 PM, Casey Stella <
>> > > > ceste...@gmail.com>
>> > > > > > wrote:
>> > > > > > >
>> > > > > > > > So, I do agree with the concern. Is there a way to host
>> the
>> > > > package
>> > > > > > > > within Metron? I definitely would like to see the
>> > > > modifications at
>> > > > > > > > https://github.com/bro/bro-plu
>> gins/commit/b9f1f35415cb0db
>> > > > > > > > 065348da0a5043a8353b4a0a8 brought back into Metron and
>> I'd
>> > > > love for
>> > > > > us
>> > > > > > to
>> > > > > > > > host the plugin.
>> > > > > > > >
>> > > > > > > > Thoughts?
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On Thu, Mar 30, 2017 at 9:09 PM, zeo...@gmail.com <
>> > > > zeo...@gmail.com>
>> > > >
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > >> Today I was taking a look at METRON-812
>> > > > > > > >> <https://issues.apache.org/jira/browse/METRON-812>,
>> which
>> > > > made me
>> > > > > > > recall
>> > > > > > > >> some conversations from a while back regarding where
>> the
>> > bro
>> > > > kafka
>> > > > > > > plugin
>> > > > > > > >> should ultimately live, and how to update it.
>> > > > > > > >>
>> > > > > > > >> Back in METRON-348 <https://issues.apache.org/
>> > > > > jira/browse/METRON-348>
>> > > > > > I
>> > > > > > > >> brought up the fact that some important changes
>> > > > > > > >> <https://github.com/bro/bro-plugins/commit/
>> > > b9f1f35415cb0db06
>> > > > > > > >> 5348da0a5043a8353b4a0a8>
>> > > > > > > >> were made to the externally hosted version of the
kafka
>> > > > plugin,
>> > > > and
>> > > > > > were
>> > > > > > > >> never introduced to Metron's hosted version (i.e. the
>> one
>> > we
>> > > > use
>> > > > > > > >> <https://github.com/apache/inc
>> ubator-metron/blob/master/
>> > > metr
>> > > > > > > >> on-deployment/roles/bro/tasks/bro-plugin-kafka.yml>
>> > > > > > > >> in vagrant when bro is installed). The conversation
>> went
>> > > down
>> > > > the
>> > > > > > route
>> > > > > > > >> of
>> > > > > > > >> discussing whether or not the bro kafka plugin code
>> should
>> > > > continue
>> > > > > to
>> > > > > > > >> live
>> > > > > > > >> in Metron in the first place. Now, with METRON-812, I
>> see
>> > us
>> > > > > further
>> > > > > > > >> muddying the waters of where to go for the right
>> plugin,
>> > as
>> > > > our
>> > > > > > version
>> > > > > > > is
>> > > > > > > >> still missing the public changes but adds some very
>> > > important
>> > > > new
>> > > > > > > >> functionality.
>> > > > > > > >>
>> > > > > > > >> I'd like to bring up the idea of using bro's packages
>> > > > > > > >> <https://github.com/bro/packages> framework, released
>> in
>> > > > late 2016
>> > > > > > > >> <
>> > > > http://blog.bro.org/2016/10/introducing-bro-package-manager.html>
>> > > > > > > >> (additional
>> > > > > > > >> documentation here <
>> > > > > > > http://bro-package-manager.readthedocs.io/en/stable/
>> > > > > > > >> >),
>> > > > > > > >> as a potential place for this to be hosted/referenced.
>> > This
>> > > > is a
>> > > > > > simple
>> > > > > > > >> and supported method (funded by Mozilla
>> > > > > > > >> <https://blog.mozilla.org/blog
>> /2015/12/10/mozilla-open-
>> > > sourc
>> > > > > > > >> e-support-first-awards-made/>)
>> > > > > > > >> to install and uninstall bro scripts, plugins, etc.,
>> and
>> > it
>> > > > also
>> > > > > > allows
>> > > > > > > us
>> > > > > > > >> to continue to have enough control over updates to the
>> > > plugin
>> > > > so
>> > > > > that
>> > > > > > it
>> > > > > > > >> will not slow down Metron development by having it as
a
>> > > > dependency
>> > > > > > > >> (resolving both of Casey's concerns noted here
>> > > > > > > >> <https://issues.apache.org/jira/browse/METRON-348?
>> > > focusedCom
>> > > > > > > >> mentId=15391865&page=com.atlassian.jira.plugin.system.
>> > > > > > > >> issuetabpanels:comment-tabpanel#comment-15391865>,
>> > > > > > > >> and I think this solution is supported by Nick's
>> comments
>> > > here
>> > > > > > > >> <https://issues.apache.org/jira/browse/METRON-348?
>> > > focusedCom
>> > > > > > > >> mentId=15391872&page=com.atlassian.jira.plugin.system.
>> > > > > > > >> issuetabpanels:comment-tabpanel#comment-15391872>
>> > > > > > > >> as
>> > > > > > > >> well).
>> > > > > > > >>
>> > > > > > > >> The only thing I'm not sure about is where to host the
>> > > plugin
>> > > > itself
>> > > > > -
>> > > > > > > my
>> > > > > > > >> first thought would be Nick's github <
>> > > > https://github.com/nickwallen
>> > > > > >,
>> > > > > > > as
>> > > > > > > >> he
>> > > > > > > >> really kicked off this effort, but maybe we can think
>> of
>> > > > something
>> > > > > > > better.
>> > > > > > > >>
>> > > > > > > >> Is this approach of interest to anybody? It is
>> extremely
>> > > > simple to
>> > > > > > put
>> > > > > > > >> together - I was able to throw one together
>> > > > > > > >> <https://github.com/bro/packages/blob/master/
>> > > > > jonzeolla/bro-pkg.index>
>> > > > > > > and
>> > > > > > > >> get it working with a fresh bro 2.5 install when
>> attending
>> > > the
>> > > > > brocon
>> > > > > > > talk
>> > > > > > > >> <
>> > https://www.bro.org/brocon2016/brocon2016_abstracts.html#
>> > > > > > > >> bro-packagemanager>
>> > > > > > > >> (recording <https://www.youtube.com/watch
>> ?v=9RFfPJeGkcE>,
>> > > > slides
>> > > > > > > >> <https://www.bro.org/brocon2016/slides/hall_bpm.pdf>)
>> > that
>> > > > > introduced
>> > > > > > > >> this
>> > > > > > > >> to me in the first place.
>> > > > > > > >>
>> > > > > > > >> Jon
>> > > > > > > >> --
>> > > > > > > >>
>> > > > > > > >> Jon
>> > > > > > > >>
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > --
>> > > > > >
>> > > > > > Jon
>> > > > > >
>> > > > >
>> > > > > --
>> > > > >
>> > > > > Jon
>> > > > >
>> > > >
>> > > > --
>> > > >
>> > > > Jon
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > >
>> > > > Jon
>> > > >
>> > > > --
>> > > >
>> > > > Jon
>> > > >
>> > > --
>> > >
>> > > Jon
>> > >
>> >
>> --
>>
>> Jon
>>
>
>
