Separate repo for MXNet infrastructure

[Yuelin Zhang]

Hello all,

I worked on a label bot which serves MXNet community. It is able to:
1. Send daily GitHub issue reports   (design
)
(PR 11861 )
2. Predict labels of unlabeled issues (design
)
(PR 11935  )
3. Help non-committers to add labels (design
)
(PR 11957 )

My concern is where should my code be finally merged. This bot is not a
part of MXNet framework but it serves MXNet community. For now, a good
option is to have a separate repo for infrastructure.

I will appreciate it if someone can share ideas.

Thanks,

Cathy

Re: Separate repo for MXNet infrastructure

[Isabel Drost-Fromm]

Am 1. August 2018 09:36:24 MESZ schrieb Yuelin Zhang 
:
>My concern is where should my code be finally merged. This bot is not a
>part of MXNet framework but it serves MXNet community. For now, a good
>option is to have a separate repo for infrastructure.

As this would be something that serves the mxnet project, my advise would be to 
make sure it ends up in a location that is controlled by the Apache mxnet PMC, 
in a location that is mirrored back to ASF resources.

Isabel

-- 
Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

Re: Separate repo for MXNet infrastructure

[Marco de Abreu]

Hi,

definitely a good point, Isabel. During our office hour we thought about
creating a repository under the Apache account with a name like
incubator-mxnet-tools or incubator-mxnet-infrastructure. Does anybody have
other ideas for naming or maybe a better solution?

Best regards,
Marco

On Wed, Aug 1, 2018 at 10:35 AM Isabel Drost-Fromm 
wrote:

>
>
> Am 1. August 2018 09:36:24 MESZ schrieb Yuelin Zhang <
> zhangyuelinch...@gmail.com>:
> >My concern is where should my code be finally merged. This bot is not a
> >part of MXNet framework but it serves MXNet community. For now, a good
> >option is to have a separate repo for infrastructure.
>
> As this would be something that serves the mxnet project, my advise would
> be to make sure it ends up in a location that is controlled by the Apache
> mxnet PMC, in a location that is mirrored back to ASF resources.
>
> Isabel
>
> --
> Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.
>

Re: Separate repo for MXNet infrastructure

[Pedro Larroy]

I like tools more.

On Wed, Aug 1, 2018 at 11:05 AM Marco de Abreu
 wrote:

> Hi,
>
> definitely a good point, Isabel. During our office hour we thought about
> creating a repository under the Apache account with a name like
> incubator-mxnet-tools or incubator-mxnet-infrastructure. Does anybody have
> other ideas for naming or maybe a better solution?
>
> Best regards,
> Marco
>
> On Wed, Aug 1, 2018 at 10:35 AM Isabel Drost-Fromm 
> wrote:
>
> >
> >
> > Am 1. August 2018 09:36:24 MESZ schrieb Yuelin Zhang <
> > zhangyuelinch...@gmail.com>:
> > >My concern is where should my code be finally merged. This bot is not a
> > >part of MXNet framework but it serves MXNet community. For now, a good
> > >option is to have a separate repo for infrastructure.
> >
> > As this would be something that serves the mxnet project, my advise would
> > be to make sure it ends up in a location that is controlled by the Apache
> > mxnet PMC, in a location that is mirrored back to ASF resources.
> >
> > Isabel
> >
> > --
> > Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.
> >
>

Re: Separate repo for MXNet infrastructure

[sandeep krishnamurthy]

apache/incubator-mxnet-tools is a good idea.
I have few queries on - how do we manage separate repository, releases,
changes by contributors may have to go to 2 repositories (when CI updates
are required), is it going to be inside 3rd party module, versions of
dependencies in tools should match with other resources in repo ex:
setup.py etc.

Why not under mxnet repo a CI / tools / infra folder and all this tools go
under it?

Best,
Sandeep

On Wed, Aug 1, 2018 at 4:12 AM Pedro Larroy 
wrote:

> I like tools more.
>
> On Wed, Aug 1, 2018 at 11:05 AM Marco de Abreu
>  wrote:
>
> > Hi,
> >
> > definitely a good point, Isabel. During our office hour we thought about
> > creating a repository under the Apache account with a name like
> > incubator-mxnet-tools or incubator-mxnet-infrastructure. Does anybody
> have
> > other ideas for naming or maybe a better solution?
> >
> > Best regards,
> > Marco
> >
> > On Wed, Aug 1, 2018 at 10:35 AM Isabel Drost-Fromm 
> > wrote:
> >
> > >
> > >
> > > Am 1. August 2018 09:36:24 MESZ schrieb Yuelin Zhang <
> > > zhangyuelinch...@gmail.com>:
> > > >My concern is where should my code be finally merged. This bot is not
> a
> > > >part of MXNet framework but it serves MXNet community. For now, a good
> > > >option is to have a separate repo for infrastructure.
> > >
> > > As this would be something that serves the mxnet project, my advise
> would
> > > be to make sure it ends up in a location that is controlled by the
> Apache
> > > mxnet PMC, in a location that is mirrored back to ASF resources.
> > >
> > > Isabel
> > >
> > > --
> > > Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.
> > >
> >
>


-- 
Sandeep Krishnamurthy

Re: Separate repo for MXNet infrastructure

[Marco de Abreu]

Very good questions! My proposal would be that projects stored in that
separate repository would be entirely independent of the mxnet code. If
there is something that would need an update on both repositories, it's a
strong indicator that it should be put into the mxnet repository. There is
no coupling between the two repositories.

>From my perspective, we would not do any releases in that repository.
Contributions would happen in the same way as it happens right now: People
make pull requests and they can then be merged by committers.

I personally think that storing things like these under the mxnet
repository might make it unnecessarily big and also create a conflict of
concerns. Code in the mxnet repository repository should be in direct
relation to the source of mxnet.
Another problem there would be our ci process. We would have to run the
entire CI pipeline although a patch might only touch the tools. This is not
necessary.

Avoiding the coupling allows us to ensure that solutions are entirely
standalone and that they build on top of the user APIs. If we tightl
integrate it, we might run into cases where people make changes in the
mxnet code out of convenience on a fly-by.

One example for a project would be the email not from Cathy. Her project is
entirely independent from the mxnet source and it should be made clear that
this is actually the case.

Best regards,
Marco

sandeep krishnamurthy  schrieb am Do., 2. Aug.
2018, 17:50:

> apache/incubator-mxnet-tools is a good idea.
> I have few queries on - how do we manage separate repository, releases,
> changes by contributors may have to go to 2 repositories (when CI updates
> are required), is it going to be inside 3rd party module, versions of
> dependencies in tools should match with other resources in repo ex:
> setup.py etc.
>
> Why not under mxnet repo a CI / tools / infra folder and all this tools go
> under it?
>
> Best,
> Sandeep
>
> On Wed, Aug 1, 2018 at 4:12 AM Pedro Larroy 
> wrote:
>
> > I like tools more.
> >
> > On Wed, Aug 1, 2018 at 11:05 AM Marco de Abreu
> >  wrote:
> >
> > > Hi,
> > >
> > > definitely a good point, Isabel. During our office hour we thought
> about
> > > creating a repository under the Apache account with a name like
> > > incubator-mxnet-tools or incubator-mxnet-infrastructure. Does anybody
> > have
> > > other ideas for naming or maybe a better solution?
> > >
> > > Best regards,
> > > Marco
> > >
> > > On Wed, Aug 1, 2018 at 10:35 AM Isabel Drost-Fromm 
> > > wrote:
> > >
> > > >
> > > >
> > > > Am 1. August 2018 09:36:24 MESZ schrieb Yuelin Zhang <
> > > > zhangyuelinch...@gmail.com>:
> > > > >My concern is where should my code be finally merged. This bot is
> not
> > a
> > > > >part of MXNet framework but it serves MXNet community. For now, a
> good
> > > > >option is to have a separate repo for infrastructure.
> > > >
> > > > As this would be something that serves the mxnet project, my advise
> > would
> > > > be to make sure it ends up in a location that is controlled by the
> > Apache
> > > > mxnet PMC, in a location that is mirrored back to ASF resources.
> > > >
> > > > Isabel
> > > >
> > > > --
> > > > Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.
> > > >
> > >
> >
>
>
> --
> Sandeep Krishnamurthy
>

Re: Separate repo for MXNet infrastructure

[Pedro Larroy]

Agree with Marco, there's a lot of stuff unrelated to MXNet. And right now
we have good separation of concerns via the dockerized builds and
ci/build.py infrastructure.
I'm also in favor of a separate repo.

On Thu, Aug 2, 2018 at 6:08 PM Marco de Abreu
 wrote:

> Very good questions! My proposal would be that projects stored in that
> separate repository would be entirely independent of the mxnet code. If
> there is something that would need an update on both repositories, it's a
> strong indicator that it should be put into the mxnet repository. There is
> no coupling between the two repositories.
>
> From my perspective, we would not do any releases in that repository.
> Contributions would happen in the same way as it happens right now: People
> make pull requests and they can then be merged by committers.
>
> I personally think that storing things like these under the mxnet
> repository might make it unnecessarily big and also create a conflict of
> concerns. Code in the mxnet repository repository should be in direct
> relation to the source of mxnet.
> Another problem there would be our ci process. We would have to run the
> entire CI pipeline although a patch might only touch the tools. This is not
> necessary.
>
> Avoiding the coupling allows us to ensure that solutions are entirely
> standalone and that they build on top of the user APIs. If we tightl
> integrate it, we might run into cases where people make changes in the
> mxnet code out of convenience on a fly-by.
>
> One example for a project would be the email not from Cathy. Her project is
> entirely independent from the mxnet source and it should be made clear that
> this is actually the case.
>
> Best regards,
> Marco
>
> sandeep krishnamurthy  schrieb am Do., 2.
> Aug.
> 2018, 17:50:
>
> > apache/incubator-mxnet-tools is a good idea.
> > I have few queries on - how do we manage separate repository, releases,
> > changes by contributors may have to go to 2 repositories (when CI updates
> > are required), is it going to be inside 3rd party module, versions of
> > dependencies in tools should match with other resources in repo ex:
> > setup.py etc.
> >
> > Why not under mxnet repo a CI / tools / infra folder and all this tools
> go
> > under it?
> >
> > Best,
> > Sandeep
> >
> > On Wed, Aug 1, 2018 at 4:12 AM Pedro Larroy <
> pedro.larroy.li...@gmail.com>
> > wrote:
> >
> > > I like tools more.
> > >
> > > On Wed, Aug 1, 2018 at 11:05 AM Marco de Abreu
> > >  wrote:
> > >
> > > > Hi,
> > > >
> > > > definitely a good point, Isabel. During our office hour we thought
> > about
> > > > creating a repository under the Apache account with a name like
> > > > incubator-mxnet-tools or incubator-mxnet-infrastructure. Does anybody
> > > have
> > > > other ideas for naming or maybe a better solution?
> > > >
> > > > Best regards,
> > > > Marco
> > > >
> > > > On Wed, Aug 1, 2018 at 10:35 AM Isabel Drost-Fromm <
> isa...@apache.org>
> > > > wrote:
> > > >
> > > > >
> > > > >
> > > > > Am 1. August 2018 09:36:24 MESZ schrieb Yuelin Zhang <
> > > > > zhangyuelinch...@gmail.com>:
> > > > > >My concern is where should my code be finally merged. This bot is
> > not
> > > a
> > > > > >part of MXNet framework but it serves MXNet community. For now, a
> > good
> > > > > >option is to have a separate repo for infrastructure.
> > > > >
> > > > > As this would be something that serves the mxnet project, my advise
> > > would
> > > > > be to make sure it ends up in a location that is controlled by the
> > > Apache
> > > > > mxnet PMC, in a location that is mirrored back to ASF resources.
> > > > >
> > > > > Isabel
> > > > >
> > > > > --
> > > > > Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail
> gesendet.
> > > > >
> > > >
> > >
> >
> >
> > --
> > Sandeep Krishnamurthy
> >
>

Re: Separate repo for MXNet infrastructure

[Marco de Abreu]

Just to clarify and avoid confusion: The dockerized pipeline and other
things that are tightly integrated would stay in the mxnet repository.

We would only add new projects to the new repository.

-Marco

Pedro Larroy  schrieb am Do., 2. Aug. 2018,
18:42:

> Agree with Marco, there's a lot of stuff unrelated to MXNet. And right now
> we have good separation of concerns via the dockerized builds and
> ci/build.py infrastructure.
> I'm also in favor of a separate repo.
>
> On Thu, Aug 2, 2018 at 6:08 PM Marco de Abreu
>  wrote:
>
> > Very good questions! My proposal would be that projects stored in that
> > separate repository would be entirely independent of the mxnet code. If
> > there is something that would need an update on both repositories, it's a
> > strong indicator that it should be put into the mxnet repository. There
> is
> > no coupling between the two repositories.
> >
> > From my perspective, we would not do any releases in that repository.
> > Contributions would happen in the same way as it happens right now:
> People
> > make pull requests and they can then be merged by committers.
> >
> > I personally think that storing things like these under the mxnet
> > repository might make it unnecessarily big and also create a conflict of
> > concerns. Code in the mxnet repository repository should be in direct
> > relation to the source of mxnet.
> > Another problem there would be our ci process. We would have to run the
> > entire CI pipeline although a patch might only touch the tools. This is
> not
> > necessary.
> >
> > Avoiding the coupling allows us to ensure that solutions are entirely
> > standalone and that they build on top of the user APIs. If we tightl
> > integrate it, we might run into cases where people make changes in the
> > mxnet code out of convenience on a fly-by.
> >
> > One example for a project would be the email not from Cathy. Her project
> is
> > entirely independent from the mxnet source and it should be made clear
> that
> > this is actually the case.
> >
> > Best regards,
> > Marco
> >
> > sandeep krishnamurthy  schrieb am Do., 2.
> > Aug.
> > 2018, 17:50:
> >
> > > apache/incubator-mxnet-tools is a good idea.
> > > I have few queries on - how do we manage separate repository, releases,
> > > changes by contributors may have to go to 2 repositories (when CI
> updates
> > > are required), is it going to be inside 3rd party module, versions of
> > > dependencies in tools should match with other resources in repo ex:
> > > setup.py etc.
> > >
> > > Why not under mxnet repo a CI / tools / infra folder and all this tools
> > go
> > > under it?
> > >
> > > Best,
> > > Sandeep
> > >
> > > On Wed, Aug 1, 2018 at 4:12 AM Pedro Larroy <
> > pedro.larroy.li...@gmail.com>
> > > wrote:
> > >
> > > > I like tools more.
> > > >
> > > > On Wed, Aug 1, 2018 at 11:05 AM Marco de Abreu
> > > >  wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > definitely a good point, Isabel. During our office hour we thought
> > > about
> > > > > creating a repository under the Apache account with a name like
> > > > > incubator-mxnet-tools or incubator-mxnet-infrastructure. Does
> anybody
> > > > have
> > > > > other ideas for naming or maybe a better solution?
> > > > >
> > > > > Best regards,
> > > > > Marco
> > > > >
> > > > > On Wed, Aug 1, 2018 at 10:35 AM Isabel Drost-Fromm <
> > isa...@apache.org>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > >
> > > > > > Am 1. August 2018 09:36:24 MESZ schrieb Yuelin Zhang <
> > > > > > zhangyuelinch...@gmail.com>:
> > > > > > >My concern is where should my code be finally merged. This bot
> is
> > > not
> > > > a
> > > > > > >part of MXNet framework but it serves MXNet community. For now,
> a
> > > good
> > > > > > >option is to have a separate repo for infrastructure.
> > > > > >
> > > > > > As this would be something that serves the mxnet project, my
> advise
> > > > would
> > > > > > be to make sure it ends up in a location that is controlled by
> the
> > > > Apache
> > > > > > mxnet PMC, in a location that is mirrored back to ASF resources.
> > > > > >
> > > > > > Isabel
> > > > > >
> > > > > > --
> > > > > > Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail
> > gesendet.
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > > Sandeep Krishnamurthy
> > >
> >
>

Re: Separate repo for MXNet infrastructure

[Pedro Larroy]

Was there any confusion?

On Thu, Aug 2, 2018 at 7:02 PM Marco de Abreu
 wrote:

> Just to clarify and avoid confusion: The dockerized pipeline and other
> things that are tightly integrated would stay in the mxnet repository.
>
> We would only add new projects to the new repository.
>
> -Marco
>
> Pedro Larroy  schrieb am Do., 2. Aug. 2018,
> 18:42:
>
> > Agree with Marco, there's a lot of stuff unrelated to MXNet. And right
> now
> > we have good separation of concerns via the dockerized builds and
> > ci/build.py infrastructure.
> > I'm also in favor of a separate repo.
> >
> > On Thu, Aug 2, 2018 at 6:08 PM Marco de Abreu
> >  wrote:
> >
> > > Very good questions! My proposal would be that projects stored in that
> > > separate repository would be entirely independent of the mxnet code. If
> > > there is something that would need an update on both repositories,
> it's a
> > > strong indicator that it should be put into the mxnet repository. There
> > is
> > > no coupling between the two repositories.
> > >
> > > From my perspective, we would not do any releases in that repository.
> > > Contributions would happen in the same way as it happens right now:
> > People
> > > make pull requests and they can then be merged by committers.
> > >
> > > I personally think that storing things like these under the mxnet
> > > repository might make it unnecessarily big and also create a conflict
> of
> > > concerns. Code in the mxnet repository repository should be in direct
> > > relation to the source of mxnet.
> > > Another problem there would be our ci process. We would have to run the
> > > entire CI pipeline although a patch might only touch the tools. This is
> > not
> > > necessary.
> > >
> > > Avoiding the coupling allows us to ensure that solutions are entirely
> > > standalone and that they build on top of the user APIs. If we tightl
> > > integrate it, we might run into cases where people make changes in the
> > > mxnet code out of convenience on a fly-by.
> > >
> > > One example for a project would be the email not from Cathy. Her
> project
> > is
> > > entirely independent from the mxnet source and it should be made clear
> > that
> > > this is actually the case.
> > >
> > > Best regards,
> > > Marco
> > >
> > > sandeep krishnamurthy  schrieb am Do., 2.
> > > Aug.
> > > 2018, 17:50:
> > >
> > > > apache/incubator-mxnet-tools is a good idea.
> > > > I have few queries on - how do we manage separate repository,
> releases,
> > > > changes by contributors may have to go to 2 repositories (when CI
> > updates
> > > > are required), is it going to be inside 3rd party module, versions of
> > > > dependencies in tools should match with other resources in repo ex:
> > > > setup.py etc.
> > > >
> > > > Why not under mxnet repo a CI / tools / infra folder and all this
> tools
> > > go
> > > > under it?
> > > >
> > > > Best,
> > > > Sandeep
> > > >
> > > > On Wed, Aug 1, 2018 at 4:12 AM Pedro Larroy <
> > > pedro.larroy.li...@gmail.com>
> > > > wrote:
> > > >
> > > > > I like tools more.
> > > > >
> > > > > On Wed, Aug 1, 2018 at 11:05 AM Marco de Abreu
> > > > >  wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > definitely a good point, Isabel. During our office hour we
> thought
> > > > about
> > > > > > creating a repository under the Apache account with a name like
> > > > > > incubator-mxnet-tools or incubator-mxnet-infrastructure. Does
> > anybody
> > > > > have
> > > > > > other ideas for naming or maybe a better solution?
> > > > > >
> > > > > > Best regards,
> > > > > > Marco
> > > > > >
> > > > > > On Wed, Aug 1, 2018 at 10:35 AM Isabel Drost-Fromm <
> > > isa...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Am 1. August 2018 09:36:24 MESZ schrieb Yuelin Zhang <
> > > > > > > zhangyuelinch...@gmail.com>:
> > > > > > > >My concern is where should my code be finally merged. This bot
> > is
> > > > not
> > > > > a
> > > > > > > >part of MXNet framework but it serves MXNet community. For
> now,
> > a
> > > > good
> > > > > > > >option is to have a separate repo for infrastructure.
> > > > > > >
> > > > > > > As this would be something that serves the mxnet project, my
> > advise
> > > > > would
> > > > > > > be to make sure it ends up in a location that is controlled by
> > the
> > > > > Apache
> > > > > > > mxnet PMC, in a location that is mirrored back to ASF
> resources.
> > > > > > >
> > > > > > > Isabel
> > > > > > >
> > > > > > > --
> > > > > > > Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail
> > > gesendet.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Sandeep Krishnamurthy
> > > >
> > >
> >
>