Hi Folks,
Have created a UDF that queries a confluent schema registry for a schema,
which is then used within a Dataset Select with the from_avro function to
decode an avro encoded value (reading from a bunch of kafka topics)
Dataset<Row> recordDF = df.select(
callUDF("getjsonSchemaUDF",col("topic").as("schema")),
from_avro(col("avro_value"),
callUDF("getjsonSchemaUDF",col("topic"))).as("record")
);
from_avro expects a String type as the second argument, and not a column
type
I need to be able convert the jsonSchema returned from Col to String type
Any ideas