Re: StructuredStreaming - processing data based on new events in Kafka topic

2022-03-13 Thread karan alang
Hi Mich, The code I sent for the function 'convertToDictForEachBatch' is not the complete code. It does use the DF to do a bunch of transformations/operations. Specific to the problem I sent the email for : One piece of the code reloads the prediction data from Bigquery based on the 'event' in

json4s parsing StructType

2022-03-13 Thread Wang Cheng
I'm trying to leverage json4s for parsing the StructType from Spark. ``` scala import org.apache.spark.sql.types._ scala case class Person(name: StructType) defined class Person scala import org.json4s._ import org.json4s._ scala import org.json4s.jackson.JsonMethods._ import