Hi Saikat, This sounds very interesting - I've been thinking about how Ignite compute engine could be enhanced, and integration with Apache Beam is one of the options I have in mind. Can you please describe how you plan to implement this? Will it run on top of the Ignite Compute Grid? How are you going to handle the failover, especially in the case of async pipeline execution?
-Val On Sat, Aug 15, 2020 at 12:50 PM Saikat Maitra <saikat.mai...@gmail.com> wrote: > Hi, > > I have been working on implementing the Apache Ignite Runner to run Apache > Beam pipeline. I have created IgniteRunner and IgnitePipelineOptions. I > have implemented the normalize pipeline method and currently working on run > method implementation for Pipeline and IgnitePipelineTranslator. > > Jira : https://issues.apache.org/jira/browse/BEAM-9045 > > PR : https://github.com/apache/beam/pull/12593 > > Please review and feel free to share any feedback or questions. > > Regards, > Saikat >
