Hi Pritam, I'm sorry that I'm not familiar with Cassandra. If your async function is always the root cause for backpressure, I think you can check the latency for the async request in your function and log some metrics.
By the way, I think you can add cache in your async function to speedup the lookup request which we always do in loopup join for sql jobs. Best, Shammon FY On Mon, Jul 17, 2023 at 10:09 PM Pritam Agarwala < pritamagarwala...@gmail.com> wrote: > Hi Team, > > > Any input on this will be really helpful. > > > Thanks! > > On Tue, Jul 11, 2023 at 12:04 PM Pritam Agarwala < > pritamagarwala...@gmail.com> wrote: > >> Hi Team, >> >> >> I am using "AsyncDataStream.unorderedWait" to connect to cassandra . The >> cassandra lookup operators are becoming the busy operator and creating >> back-pressure result low throughput. >> >> >> The Cassandra lookup is a very simple query. So I increased the capacity >> parameter to 80 from 15 and could see low busy % of cassandra operators. I >> am monitoring the cassandra open connections and connected host metrics. >> Couldn't see any change on these metrics. >> >> >> How is the capacity parameter related to cassandra open connections and >> host ? If I increase capacity more will it have any impact on these metrics >> ? >> >> Thanks & Regards, >> Pritam >> >
