Re: Apache Ignite as a distributed processing back-ends

Those are the valid points. Personally, I would go for the beam-repo way because it will guarantee that the integration works as expected with every Beam release. This is for instance how Ignite is integrated with Camel, MyBatis, Zeppelin. Anyway, here is a ticket. Hope that an Ignite fellow

Re: Apache Ignite as a distributed processing back-ends

Hosting integrations in impl more than beam makes a lot of sense IMHO while you can maintain it and follow beam release cycle. It will enable you to evolve faster and optimise/adapt it more accurately. If you dont have the resources, beam would fit better and guarantee it works with each release.

Re: Apache Ignite as a distributed processing back-ends

Having it inside the Apache Beam repo makes sense and I could see it being a good fit as an IO and as a runner. On Tue, Dec 12, 2017 at 3:29 PM, Denis Magda wrote: > Hi Romain, > > Thanks for the reference. Do you prefer to have the Ignite runner in > Beam’s code base? > >

Re: Apache Ignite as a distributed processing back-ends

Hi Romain, Thanks for the reference. Do you prefer to have the Ignite runner in Beam’s code base? From what I see, the current runners are hosted there: https://github.com/apache/beam/tree/master/runners As for Ignite community, we would

Re: Apache Ignite as a distributed processing back-ends

Hi This sounds awesome to have an Ignite runner which could compete with hazelcast-jet. The entry point would be https://beam.apache.org/contribute/runner-guide/ IMHO. Being on Ignite cluster also opens a lot of doors - reusing the filesystem or distributed structures. Very exiting. Le 8 déc.

Apache Ignite as a distributed processing back-ends

Hello Apache Beam fellows! We at Apache Ignite community came across your project and would be happy to integrate with it. In short, Ignite is a distributed database and computational platform that has its own map-reduce like component: https://apacheignite.readme.io/docs/compute-grid