here is the stackoverflow link
https://stackoverflow.com/questions/73780259/spark-structured-streaming-stderr-getting-filled-up
On Mon, Sep 19, 2022 at 4:41 PM karan alang wrote:
> I've created a stackoverflow ticket for this as well
>
> On Mon, Sep 19, 2022 at 4:37 PM karan ala
I've created a stackoverflow ticket for this as well
On Mon, Sep 19, 2022 at 4:37 PM karan alang wrote:
> Hello All,
> I've a Spark Structured Streaming job on GCP Dataproc - which picks up
> data from Kafka, does processing and pushes data back into kafka topics.
>
> Couple of questions :
> 1.
Hello All,
I've a Spark Structured Streaming job on GCP Dataproc - which picks up data
from Kafka, does processing and pushes data back into kafka topics.
Couple of questions :
1. Does Spark put all the log (incl. INFO, WARN etc) into stderr ?
What I notice is that stdout is empty, while all the
