There're sections in SS programming guide which exactly answer these
questions:
http://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#managing-streaming-queries
http://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#monitoring-streaming-queries
In 3.0 the community just added it.
On Sun, 5 Jul 2020, 14:28 KhajaAsmath Mohammed,
wrote:
> Hi,
>
> We are trying to move our existing code from spark dstreams to structured
> streaming for one of the old application which we built few years ago.
>
> Structured streaming job doesn’t have
Thank you all for the responses. I believe the user shouldn't be
worried about creating the log dir explicitly. The event logging should
behave like other logs (e.g. master or slave) that the directory should
be automatically created if not exist.
-- ND
On 7/2/20 9:19 AM, Zero wrote:
Hi,
We are trying to move our existing code from spark dstreams to structured
streaming for one of the old application which we built few years ago.
Structured streaming job doesn’t have streaming tab in sparkui. Is there a way
to monitor the job submitted by us in structured streaming ? Since
