Re: Issue with back pressure and AsyncFunction

2017-11-15 Thread Aljoscha Krettek
Hi, Unfortunately, I don't have anything to add. Yes, back pressure doesn't work correctly for functions that do work outside the main thread and iterations currently don't work well and can lead to deadlocks. Did you already open issues for those by now? Best, Aljoscha > On 10. Nov 2017, at

Re: Issue with back pressure and AsyncFunction

2017-11-10 Thread Ufuk Celebi
Hey Ken, thanks for your message. Both your comments are correct (see inline). On Fri, Nov 10, 2017 at 10:31 PM, Ken Krugler wrote: > 1. A downstream function in the iteration was (significantly) increasing the > number of tuples - it would get one in, and sometimes

Issue with back pressure and AsyncFunction

2017-11-10 Thread Ken Krugler
Hi all, I was debugging a curious problem with a streaming job that contained an iteration and several AsynFunctions. The entire job would stall out, with no progress being made. But when I checked back pressure, only one function showed it as being high - everything else was OK. And when I