Thanks Kevin,
Worked like a charm.
FYI for readers,
val temp1 =
temp.groupBy("reference_id").pivot("char_name").agg(max($"char_value"))
I didn't know I can use 'agg' with a string max. I was using it incorrectly
as below
temp.groupBy("reference_id").pivot("char_name").max("char_value")
On Wed, Feb 1, 2017 at 11:56 PM, Kevin Mellott <kevin.r.mell...@gmail.com>
wrote:
> This should work for non-numerical data as well - can you please elaborate
> on the error you are getting and provide a code sample? As a preliminary
> hint, you can "aggregate" text values using *max*.
>
> df.groupBy("someCol")
> .pivot("anotherCol")
> .agg(max($"textCol"))
>
> Thanks,
> Kevin
>
> On Wed, Feb 1, 2017 at 2:02 PM, Darshan Pandya <darshanpan...@gmail.com>
> wrote:
>
>> Hello,
>>
>> I am trying to transpose some data using groupBy pivot aggr as mentioned
>> in this blog
>> https://databricks.com/blog/2016/02/09/reshaping-data-with-
>> pivot-in-apache-spark.html
>>
>> But this works only for numerical data.
>> Any hints for doing the same thing for non numerical data ?
>>
>>
>> --
>> Sincerely,
>> Darshan
>>
>>
>
--
Sincerely,
Darshan
