Hi there, I have two dataframes that each have some columns which are of list type (array<int> generated by the collect_list function actually).
I need to outer join these two dfs, however by nature of an outer join I am sometimes left with null values. Normally I would use df.na.fill(...), however it appears the fill function doesn't support this data type. Can anyone recommend an alternative? I have also been playing around with coalesce in a sql expression, but I'm not having any luck here either. Obviously, I can do a null check on the fields downstream, however it is not in the spirit of scala to pass around nulls, so I wanted to see if I was missing another approach first. Thanks, Sumona I am using Spark 2.0.2
