small correction.
If I try to convert a Row into a Json String it results into something like
this {"key1", "name", "value1": "hello", "key2", "ratio", "value2": 1.56 ,
"key3", "count", "value3": 34} but *what I need is something like this {
result: {"name": "hello", "ratio": 1.56, "count": 34} } however I don't
have a result column or even this {**"name": "hello", "ratio": 1.56,
"count": 34} **would work.*
On Wed, May 31, 2017 at 1:05 AM, kant kodali <kanth...@gmail.com> wrote:
> Hi Jules,
>
> I read that blog several times prior to asking this question.
>
> Thanks!
>
> On Wed, May 31, 2017 at 12:12 AM, Jules Damji <dmat...@comcast.net> wrote:
>
>> Hello Kant,
>>
>> See is the examples in this blog explains how to deal with your
>> particular case: https://databricks.com/blog/2017/02/23/working-complex
>> -data-formats-structured-streaming-apache-spark-2-1.html
>>
>> Cheers
>> Jules
>>
>> Sent from my iPhone
>> Pardon the dumb thumb typos :)
>>
>> On May 30, 2017, at 7:31 PM, kant kodali <kanth...@gmail.com> wrote:
>>
>> Hi All,
>>
>> I have a Dataset<Row> and I am trying to convert it into Dataset<String>
>> (json String) using Spark Structured Streaming. I have tried the following.
>>
>> df2.toJSON().writeStream().foreach(new KafkaSink())
>>
>> This doesn't seem to work for the following reason.
>>
>> "Queries with streaming sources must be executed with writeStream.start()"
>>
>> My dataframe has looks like this
>>
>> name, ratio, count // column names
>>
>> "hello", 1.56, 34
>>
>> If I try to convert a Row into a Json String it results into something
>> like this {"key1", "name", "value1": "hello", "key2", "ratio", "value2":
>> 1.56 , "key3", "count", "value3": 34} but *what I need is something like
>> this { result: {"name": "hello", "ratio": 1.56, "count": 34} } however I
>> don't have a result column. *
>>
>> It looks like there are couple of functions to_json and json_tuple but
>> they seem to take only one Column as a first argument so should I call
>> to_json on every column? Also how would I turn this into DataSet<String> ?
>>
>> Thanks!
>>
>>
>>
>>
>>
>
