Re: No. of active states?

uide.html#monitoring-streaming-queries Hope this helps. On Fri, May 8, 2020 at 3:26 AM Something Something wrote: Is there a way to get the total no. of active states in memory at any given point in a Stateful Spark Structured Streaming job? We are thinking of using this metric for 'Auto Scaling' our Spark cluster.

Re: No. of active states?

, it's being >> provided via StreamingQueryListener. >> >> >> http://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#monitoring-streaming-queries >> >> Hope this helps. >> >> On Fri, May 8, 2020 at 3:26 AM Something Something

Re: No. of active states?

's being > provided via StreamingQueryListener. > > > http://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#monitoring-streaming-queries > > Hope this helps. > > On Fri, May 8, 2020 at 3:26 AM Something Something < > mailinglist...@gmail.com> w

Re: No. of active states?

ing wrote: > Is there a way to get the total no. of active states in memory at any > given point in a Stateful Spark Structured Streaming job? We are thinking > of using this metric for 'Auto Scaling' our Spark cluster. >

No. of active states?

Is there a way to get the total no. of active states in memory at any given point in a Stateful Spark Structured Streaming job? We are thinking of using this metric for 'Auto Scaling' our Spark cluster.