json4s parsing StructType

2022-03-13 Thread Wang Cheng
I'm trying to leverage json4s for parsing the StructType from Spark. ``` scala> import org.apache.spark.sql.types._ scala> case class Person(name: StructType) defined class Person scala> import org.json4s._ import org.json4s._ scala> import org.json4s.jackson.JsonMethods._ import org.json4s.j

Re: StructuredStreaming - processing data based on new events in Kafka topic

2022-03-13 Thread karan alang
Hi Mich, The code I sent for the function 'convertToDictForEachBatch' is not the complete code. It does use the DF to do a bunch of transformations/operations. Specific to the problem I sent the email for : One piece of the code reloads the prediction data from Bigquery based on the 'event' in to