Re: Multi threaded operators?

2018-04-23 Thread Ken Krugler
Hi Alex, Given that you’re hitting a DB, the approach of using multi-threaded access from a CoFlatMapFunction or AsyncFunction makes sense - you don’t want to try to abuse Flink’s parallelism. I’ve done it both ways, so either is an option. If you use an AsyncFunction, you get the benefit of

Re: Multi threaded operators?

2018-04-23 Thread Lasse Nedergaard
Hi. What kind of problems and what configuration should we be aware of.? Med venlig hilsen / Best regards Lasse Nedergaard > Den 23. apr. 2018 kl. 13.44 skrev Jörn Franke : > > I would disable it if possible and use the Flink parallism. The threading > might work but

Re: Multi threaded operators?

2018-04-23 Thread Jörn Franke
I would disable it if possible and use the Flink parallism. The threading might work but can create operational issues depending on how you configure your resource manager. > On 23. Apr 2018, at 11:54, Alexander Smirnov > wrote: > > Hi, > > I have a co-flatmap

Multi threaded operators?

2018-04-23 Thread Alexander Smirnov
Hi, I have a co-flatmap function which reads data from external DB on specific events. The API for the DB layer is homegrown and it uses multiple threads to speed up reading. Can it cause any problems if I use the multithreading API in the flatmap1 function? Is it allowed in Flink? Or, maybe I