Hello all, I have this peculiar problem where quote " characters are added to the beginning and end of my string values.
I get data using Structured Streaming from an Azure Event Hub using a Scala Notebook in Azure Databricks. The Dataframe schema received contain a property of type Map named "properties" containing string/string key value pairs. Only one pair in my case. I input data from a C# program where the "properties" property in the EventData schema is a Dictionary<string, object> where I input a string/string property pair. If I input a string value of _Hello_ in C# I get back a string value of _"Hello"_ in my Dataframe. In the body tag of the Schema I input a UTF-8 byte[] and get back a UTF-8 byte[] which I cast to string, this turns out correct, no " are added to the string value. If I try to pass in a UTF-8 string as a byte[] as the value of the KvP I get a serialization exception so that's a no go. Any idea on why or where the quote characters "" are added to the value of the string when I read them in Spark? Any ideas would greatly be appreciated. //Magnus
