Hello!
Is it possible to write pyspark UDF, generated data to streaming dataframe?
I want to get some data from REST API requests in real time and consider to
save this data to dataframe.
And then put it to Kafka.
I can't realise how to create streaming dataframe from generated data.
I am new in
otikov
From: Mich Talebzadeh
Sent: Wednesday, December 27, 2023 6:17 PM
To: Поротиков Станислав Вячеславович
Cc: user@spark.apache.org
Subject: Re: Pyspark UDF as a data source for streaming
Ok so you want to generate some random data and load it into Kafka on a regular
interval and the rest
From: Mich Talebzadeh
Sent: Wednesday, December 27, 2023 6:17 PM
To: Поротиков Станислав Вячеславович
Cc: user@spark.apache.org
Subject: Re: Pyspark UDF as a data source for streaming
Ok so you want to generate some random data and load it into Kafka on a regular
interval and the rest
author
will in no case be liable for any monetary damages arising from such loss,
damage or destruction.
On Wed, 27 Dec 2023 at 13:26, Поротиков Станислав Вячеславович
mailto:s.poroti...@skbkontur.ru>> wrote:
Actually it's json with specific structure from API server.
But the task is to chec
Ok. Thank you very much!
Best regards,
Stanislav Porotikov
From: Mich Talebzadeh
Sent: Thursday, December 28, 2023 5:14 PM
To: Hyukjin Kwon
Cc: Поротиков Станислав Вячеславович ;
user@spark.apache.org
Subject: Re: Pyspark UDF as a data source for streaming
You can work around this issue
