"ignite-extensions" might be a better name in the long term if we decide to place any extension to the repo that doesn't integrate with any 3rd party technology but rather enhances Ignite with extra API.
After thinking about it for the last few days, I think that the best approach to start with should be as follows: 1. Let's create an "ignite-extensions" repo and let Emmanouil contribute the Pub/Sub Streamer integration there. 2. Let's move after or in parallel our streaming integrations there like Flink, Camel, MQTT, etc.: https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations 3. After checking the approach with the first integrations we can carry on with the rest of modules: Spark and Hibernate require their own repos, our TensorFlow integration might be moved to "ignite-extensions" as well unless Alexey Zinoviev suggests a dedicated extensions repo for ML integrations. Thoughts? Saikat, are you willing to help with #1 and lead #2? - Denis On Tue, Oct 29, 2019 at 1:25 AM Ivan Pavlukhin <vololo...@gmail.com> wrote: > Folks, > > Just another one name candidate "ignite-integrations" > > Or are there not only integrations? > > вт, 29 окт. 2019 г. в 03:31, Saikat Maitra <saikat.mai...@gmail.com>: > > > > Hi Denis, > > > > I meant we can create dedicated repository for individual extensions and > we > > can create submodules inside each specific repository for individual > > extensions versions. > > > > Regards, > > Saikat > > > > > > On Mon, Oct 28, 2019 at 7:14 PM Saikat Maitra <saikat.mai...@gmail.com> > > wrote: > > > > > Hi Denis, > > > > > > Yes, dedicated repository is also a good idea and we can create > submodules > > > inside each specific repository for individual extensions. > > > > > > Regards, > > > Saikat > > > > > > > > > > > > On Mon, Oct 28, 2019 at 2:34 PM Denis Magda <dma...@apache.org> wrote: > > > > > >> Folks, > > >> > > >> What do you think about having a dedicated repository for each > extension > > >> instead of a single one? It sounds complicated but might be > worthwhile for > > >> cases when an integration requires to support several versions. Take > Spark > > >> or Hibernate as an example, we already provide different versions of > > >> Ignite > > >> packages for various Spark/Hibernate versions. It will be easier to > handle > > >> if Spark or Hibernate had their own repos. > > >> > > >> - > > >> Denis > > >> > > >> > > >> On Sat, Oct 26, 2019 at 10:06 PM Saikat Maitra < > saikat.mai...@gmail.com> > > >> wrote: > > >> > > >> > Hello Alexey, > > >> > > > >> > Thank you for your email. Yes, I am also aligned for > ignite-extensions. > > >> > > > >> > If others are also ok we can create the new repository with name > > >> > ignite-extensions. > > >> > > > >> > Regards, > > >> > Saikat > > >> > > > >> > On Sat, Oct 26, 2019 at 3:11 AM Alexey Zinoviev < > zaleslaw....@gmail.com > > >> > > > >> > wrote: > > >> > > > >> > > Vote for ignite-extensions (it's more widely than > ignite-connectors) > > >> > > > > >> > > сб, 26 окт. 2019 г. в 05:52, Saikat Maitra < > saikat.mai...@gmail.com>: > > >> > > > > >> > > > Hello Denis, Dmitriy > > >> > > > > > >> > > > Thank you for your reply. I am thinking when a new repo is > created > > >> that > > >> > > > will not mean that the project will undergo Incubator process , > Is > > >> > > > this correct? > > >> > > > > > >> > > > We actually want to just take few of our integrations and > release > > >> them > > >> > > > separately as part of this new git repository. We will continue > to > > >> keep > > >> > > the > > >> > > > release group name and artifact name same. > > >> > > > > > >> > > > I will definitely help drive the initiative to migrate our > > >> integrations > > >> > > > into this new Ignite Extension repo. > > >> > > > > > >> > > > I seek help from our community to propose new name for Ignite > > >> Extension > > >> > > > repository. > > >> > > > > > >> > > > Some ideas, I have are as follows: > > >> > > > > > >> > > > > > >> > > > - ignite-extensions > > >> > > > - ignite-sidecar > > >> > > > - ignite-ext > > >> > > > - ignite-connectors > > >> > > > > > >> > > > > > >> > > > Regards, > > >> > > > Saikat > > >> > > > > > >> > > > On Fri, Oct 25, 2019 at 4:06 PM Dmitriy Pavlov < > dpav...@apache.org> > > >> > > wrote: > > >> > > > > > >> > > > > Hi Denis, > > >> > > > > > > >> > > > > any PMC member can create repository here > > >> > > > > https://selfserve.apache.org/ > > >> > > > > > > >> > > > > We just need to select repo names in advance. > > >> > > > > > > >> > > > > Sincerely, > > >> > > > > Dmitriy Pavlov > > >> > > > > > > >> > > > > пт, 25 окт. 2019 г. в 23:57, Denis Magda <dma...@apache.org>: > > >> > > > > > > >> > > > > > Saikat, sounds great. > > >> > > > > > > > >> > > > > > As long as you're already driving this initiative, could you > > >> please > > >> > > > check > > >> > > > > > with ASF Infra how to put this approach in place? Will we be > > >> able > > >> > to > > >> > > > > create > > >> > > > > > several additional repositories for extensions and release > them > > >> as > > >> > > part > > >> > > > > of > > >> > > > > > the Ignite project? > > >> > > > > > > > >> > > > > > > > >> > > > > > - > > >> > > > > > Denis > > >> > > > > > > > >> > > > > > > > >> > > > > > On Thu, Oct 24, 2019 at 7:18 PM Saikat Maitra < > > >> > > saikat.mai...@gmail.com > > >> > > > > > > >> > > > > > wrote: > > >> > > > > > > > >> > > > > > > Hi Denis, > > >> > > > > > > > > >> > > > > > > Yes, I see it now as I think more about it, I like option > - > > >> 0. I > > >> > am > > >> > > > > > > thinking from a new member and a contributor perspective > for > > >> > Ignite > > >> > > > > > > project, I would very much like my PR and proposal to be > > >> accepted > > >> > > > > within > > >> > > > > > > Ignite community. > > >> > > > > > > > > >> > > > > > > Regards, > > >> > > > > > > Saikat > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > On Thu, Oct 24, 2019 at 1:56 PM Denis Magda < > > >> dma...@apache.org> > > >> > > > wrote: > > >> > > > > > > > > >> > > > > > > > Exactly, Dmitry found the right word for the reason why > > >> > option-0 > > >> > > > > might > > >> > > > > > be > > >> > > > > > > > the best one - to avoid the Ignite community *split*. > All > > >> those > > >> > > who > > >> > > > > > will > > >> > > > > > > be > > >> > > > > > > > contributing to integrations and extensions need to be > able > > >> to > > >> > > > earn a > > >> > > > > > > > status of an Ignite committer and PMC members. > Otherwise, > > >> I'm > > >> > not > > >> > > > > sure > > >> > > > > > if > > >> > > > > > > > Emmanouil and others to come will be fully involved in > the > > >> > Ignite > > >> > > > > > > > community. > > >> > > > > > > > > > >> > > > > > > > Saikat, Emmanouil, what do you think if to look from > this > > >> > > > > perspective? > > >> > > > > > > > > > >> > > > > > > > - > > >> > > > > > > > Denis > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > On Thu, Oct 24, 2019 at 12:04 AM Dmitriy Pavlov < > > >> > > > dpav...@apache.org> > > >> > > > > > > > wrote: > > >> > > > > > > > > > >> > > > > > > > > Hah, IMHO, it is a story of how pushing others to > place > > >> their > > >> > > > > > > > contribution > > >> > > > > > > > > outside ASF could lead projects to split their > > >> communities. > > >> > > > > > > > > > > >> > > > > > > > > I believe, the Ignite community is more open and > flexible > > >> in > > >> > > that > > >> > > > > > > regard. > > >> > > > > > > > > So Option-0. is also OK from my perspective. > > >> > > > > > > > > > > >> > > > > > > > > чт, 24 окт. 2019 г. в 04:01, Saikat Maitra < > > >> > > > > saikat.mai...@gmail.com > > >> > > > > > >: > > >> > > > > > > > > > > >> > > > > > > > > > Hi, > > >> > > > > > > > > > > > >> > > > > > > > > > I looked into the way Apache Bahir manages their > > >> extensions > > >> > > for > > >> > > > > > Spark > > >> > > > > > > > and > > >> > > > > > > > > > Flink and it looks like they are much independent in > > >> terms > > >> > of > > >> > > > > > > managing > > >> > > > > > > > > > their releases. They also have separate git repos > for > > >> > apache > > >> > > > > bahir > > >> > > > > > > and > > >> > > > > > > > > > apache bahir-flink. > > >> > > > > > > > > > > > >> > > > > > > > > > Releases : > > >> > > > > > > > > > https://bahir.apache.org/downloads/spark/ > > >> > > > > > > > > > https://bahir.apache.org/downloads/flink/ > > >> > > > > > > > > > > > >> > > > > > > > > > Repos : > > >> > > > > > > > > > https://github.com/apache/bahir > > >> > > > > > > > > > https://github.com/apache/bahir-flink > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > I am thinking if we are following the similar > pattern we > > >> > can > > >> > > > > > create a > > >> > > > > > > > > > separate git repo under the Org apache / > > >> ignite-extentions > > >> > or > > >> > > > > > apache > > >> > > > > > > / > > >> > > > > > > > > > bahir-ignite. > > >> > > > > > > > > > > > >> > > > > > > > > > If most of our integrations are data streaming > > >> connectors > > >> > > that > > >> > > > we > > >> > > > > > are > > >> > > > > > > > > most > > >> > > > > > > > > > interested to migrate to separate repository then > > >> joining > > >> > > > Apache > > >> > > > > > > Bahir > > >> > > > > > > > > > project and managing independent release cycle will > > >> benefit > > >> > > us > > >> > > > as > > >> > > > > > it > > >> > > > > > > > will > > >> > > > > > > > > > help foster cross community engagement and support. > The > > >> > > purpose > > >> > > > > of > > >> > > > > > > > Bahir > > >> > > > > > > > > is > > >> > > > > > > > > > also to host such extensions as ours. > > >> > > > > > > > > > > > >> > > > > > > > > > I was reading this news article and it resonated > similar > > >> > > ideas > > >> > > > > that > > >> > > > > > > we > > >> > > > > > > > > have > > >> > > > > > > > > > specific to managing release cycles > > >> > > > > > > > > > > > >> > > > > > > > >> > > https://thenewstack.io/apache-bahir-gives-spark-extensions-new-home/ > > >> > > > > > > > > > > > >> > > > > > > > > > Please review and share your feedback. > > >> > > > > > > > > > > > >> > > > > > > > > > Warm Regards, > > >> > > > > > > > > > Saikat > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > On Wed, Oct 23, 2019 at 4:29 PM Denis Magda < > > >> > > dma...@apache.org > > >> > > > > > > >> > > > > > > wrote: > > >> > > > > > > > > > > > >> > > > > > > > > > > Folks, > > >> > > > > > > > > > > > > >> > > > > > > > > > > How about considering the option Dmitriy named as > "0. > > >> > > placing > > >> > > > > > > > > integration > > >> > > > > > > > > > > in a separate module within space of Apache > Ignite"? > > >> > > > > > > > > > > > > >> > > > > > > > > > > Nothing prevents us from following concepts of > Bahir > > >> > > project > > >> > > > in > > >> > > > > > the > > >> > > > > > > > > sense > > >> > > > > > > > > > > that we'll be creating and managing separate > > >> repositories > > >> > > for > > >> > > > > > > Ignite > > >> > > > > > > > > > > extensions/modules but those will be governed by > the > > >> > Ignite > > >> > > > > > > community > > >> > > > > > > > > and > > >> > > > > > > > > > > all the contributors to the extensions will be > > >> becoming > > >> > > > Ignite > > >> > > > > > > > > committers > > >> > > > > > > > > > > and PMC members. The more I think about this > approach > > >> the > > >> > > > more > > >> > > > > I > > >> > > > > > > like > > >> > > > > > > > > it. > > >> > > > > > > > > > > Any thoughts? > > >> > > > > > > > > > > > > >> > > > > > > > > > > - > > >> > > > > > > > > > > Denis > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > On Wed, Oct 23, 2019 at 12:42 PM Dmitriy Pavlov < > > >> > > > > > > dpav...@apache.org> > > >> > > > > > > > > > > wrote: > > >> > > > > > > > > > > > > >> > > > > > > > > > > > Hi, Saikat, Alexey, > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > Actually we have 3 ways to solve it. > > >> > > > > > > > > > > > 0. placing integration in a separate module > within > > >> > space > > >> > > of > > >> > > > > > > Apache > > >> > > > > > > > > > Ignite > > >> > > > > > > > > > > > 1. Apache Bahir > > >> > > > > > > > > > > > 2. Apache Incubator > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > I'm not sure if option 2 is the best one since > it is > > >> > more > > >> > > > > about > > >> > > > > > > > > > building > > >> > > > > > > > > > > a > > >> > > > > > > > > > > > new community around Ignite Extensions, it may > be > > >> > tricky. > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > But 0 and 1 seem to be perfectly OK. > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > And I like option 1 most since it is very > natural to > > >> > move > > >> > > > > > > > > Ignite-Kafka, > > >> > > > > > > > > > > > Ignite-Camel to a separate project specially > > >> intended > > >> > for > > >> > > > > > > > > integration. > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > So if we stay with option 1 I would be glad to > help. > > >> > > Count > > >> > > > on > > >> > > > > > my > > >> > > > > > > > > > support > > >> > > > > > > > > > > > within the migration to Apache Bahir. > Inter-project > > >> > > > > interaction > > >> > > > > > > and > > >> > > > > > > > > > > > integration are usually welcomed in the ASF. > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > Sincerely, > > >> > > > > > > > > > > > Dmitriy Pavlov > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > ср, 23 окт. 2019 г. в 09:31, Alexey Zinoviev < > > >> > > > > > > > zaleslaw....@gmail.com > > >> > > > > > > > > >: > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > Also, dear Saikat Maitra, could you please > > >> describe > > >> > how > > >> > > > you > > >> > > > > > see > > >> > > > > > > > the > > >> > > > > > > > > > > > > release cycles in Bahir Ignite Extensions and > how > > >> it > > >> > be > > >> > > > > > related > > >> > > > > > > > to > > >> > > > > > > > > > > Ignite > > >> > > > > > > > > > > > > release, 2.9, 3.0 for example. > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > Thank you for your energy > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > ср, 23 окт. 2019 г., 8:10 Alexey Zinoviev < > > >> > > > > > > > zaleslaw....@gmail.com > > >> > > > > > > > > >: > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > >> Please, give me permissions too, I'd glad to > help > > >> > with > > >> > > > > this > > >> > > > > > > > > modules > > >> > > > > > > > > > > > >> migration and support part of them in > future, but > > >> > also > > >> > > > we > > >> > > > > > need > > >> > > > > > > > not > > >> > > > > > > > > > > only > > >> > > > > > > > > > > > >> contributor but a few Committer permissions > to > > >> merge > > >> > > In > > >> > > > > > > > repository > > >> > > > > > > > > > in > > >> > > > > > > > > > > > other > > >> > > > > > > > > > > > >> side it could be very long proccess. > > >> > > > > > > > > > > > >> > > >> > > > > > > > > > > > >> Could you ask Bahir Community about that? > > >> > > > > > > > > > > > >> > > >> > > > > > > > > > > > >> ср, 23 окт. 2019 г., 2:31 Saikat Maitra < > > >> > > > > > > > saikat.mai...@gmail.com > > >> > > > > > > > > >: > > >> > > > > > > > > > > > >> > > >> > > > > > > > > > > > >>> Hi, > > >> > > > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> I discussed with Apache Bahir community and > they > > >> > are > > >> > > > > > > interested > > >> > > > > > > > > to > > >> > > > > > > > > > > have > > >> > > > > > > > > > > > >>> Apache Ignite extensions as part of Apache > Bahir > > >> > > > project. > > >> > > > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> I have also requested for contributor > access in > > >> > Jira > > >> > > > for > > >> > > > > > > Apache > > >> > > > > > > > > > Bahir > > >> > > > > > > > > > > > >>> project so that I can create issues and > assign > > >> to > > >> > > > > myself. I > > >> > > > > > > can > > >> > > > > > > > > > help > > >> > > > > > > > > > > > with > > >> > > > > > > > > > > > >>> code reviews as well. > > >> > > > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> Also my thoughts on releases specific to > > >> > dependencies > > >> > > > for > > >> > > > > > > > Apache > > >> > > > > > > > > > > Ignite > > >> > > > > > > > > > > > >>> is > > >> > > > > > > > > > > > >>> to do a fast follow up release for modules > > >> based on > > >> > > > > latest > > >> > > > > > > > Apache > > >> > > > > > > > > > > > Ignite > > >> > > > > > > > > > > > >>> stable release. > > >> > > > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> Here is the email thread for reference > > >> > > > > > > > > > > > >>> > > >> > > > > > > > > > >> > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html > > >> > > > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> I wanted to connect and get feedback on the > > >> > proposal > > >> > > > and > > >> > > > > if > > >> > > > > > > we > > >> > > > > > > > > are > > >> > > > > > > > > > ok > > >> > > > > > > > > > > > to > > >> > > > > > > > > > > > >>> move the following Apache Ignite Extensions > > >> > > > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations > > >> > > > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> Regards, > > >> > > > > > > > > > > > >>> Saikat > > >> > > > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> On Fri, Oct 18, 2019 at 9:44 PM Saikat > Maitra < > > >> > > > > > > > > > > saikat.mai...@gmail.com > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > >>> wrote: > > >> > > > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> > Hello, > > >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > We wanted to discuss on a proposal to > move and > > >> > > > support > > >> > > > > > the > > >> > > > > > > > > Apache > > >> > > > > > > > > > > > >>> Ignite > > >> > > > > > > > > > > > >>> > integrations as separate Ignite > Extensions as > > >> > > > discussed > > >> > > > > > > here > > >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Pub-Sub-Streamer-Implementation-td43944.html > > >> > > > > > > > > > > > >>> > . > > >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > The reason we wanted to move our Apache > Ignite > > >> > > > > > integration > > >> > > > > > > as > > >> > > > > > > > > > > > separate > > >> > > > > > > > > > > > >>> > Extensions is this will help us to manage > and > > >> > > > maintain > > >> > > > > > > > separate > > >> > > > > > > > > > > > >>> lifecycle > > >> > > > > > > > > > > > >>> > for Apache Ignite integrations. > > >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > All the integrations will continue to be > part > > >> of > > >> > > ASF > > >> > > > > and > > >> > > > > > we > > >> > > > > > > > > will > > >> > > > > > > > > > > > keep > > >> > > > > > > > > > > > >>> > supporting and developing in accordance > with > > >> ASF > > >> > > > vision > > >> > > > > > and > > >> > > > > > > > > > > > practices. > > >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > We are considering following two choices > for > > >> > moving > > >> > > > to > > >> > > > > > > Apache > > >> > > > > > > > > > > Ignite > > >> > > > > > > > > > > > >>> > Extensions: > > >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > 1. Reach out to Apache Bahir community and > > >> > propose > > >> > > to > > >> > > > > > make > > >> > > > > > > > > Ignite > > >> > > > > > > > > > > > >>> > Extensions a separate module as part of > Apache > > >> > > Bahir > > >> > > > > > > project. > > >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > https://bahir.apache.org/ > > >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces96 > > >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > 2. Reach out to Apache Incubator > community and > > >> > > > request > > >> > > > > > for > > >> > > > > > > a > > >> > > > > > > > > new > > >> > > > > > > > > > > > >>> project > > >> > > > > > > > > > > > >>> > for Ignite Extensions. > > >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > Please review and share feedback on our > > >> proposal. > > >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > Warm Regards, > > >> > > > > > > > > > > > >>> > Saikat > > >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > > >> > > > > > > > > > > > >> > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > > > -- > Best regards, > Ivan Pavlukhin >
